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A machine vision method analyzes a cali- 
bration target of the type having two or more re- 
gions, each having a 'imageable characteristic", 
e.g., a different color, contrast, or brightness, 
from its neighboring rcgion(s). Each region has 
at least two edges - referred to as "adjoining 
edges" - that are linear and that are directed to- 
ward and, optionally meet at, a reference point 
(e.g., the center of the target or some other lo- 
cation of interest). The method includes gener- 
ating an image of the target, identifying in the 
image features corresponding to the adjoining 
edges, fitting lines to those edges, and determin- 
ing the orientation and/or position of the target 
from those lines. 
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MACHINE VISION CALIBRATION TARGETS AND METHODS 
OF DETERMINING THEIR LOCATION AND ORIENTATION IN AN IMAGE 

Reservation of Copyright 

The disclosure of this patent document contains material which is subject to 
copyright protection. The owner thereof has no objection to facsimile reproduction by 
anyone of the patent document or the patent disclosure, as it appears in the U.S. Patent and 
Trademark Office patent file or records, but otherwise reserves all copyright rights 
whatsoever. 

Background of the Invention 

The invention pertains to machine vision and, more particularly, to calibration 
targets and methods for determining their location and orientation in an image. 

Machine vision refers to the automated analysis of an image to determine 
characteristics of objects and other features shown in the image. It is often employed in 
automated manufacturing lines, where images of components are analyzed to determine 
placement and alignment prior to assembly. Machine vision is also used for quality 
assurance. For example, in the pharmaceutical and food packing industries, images of 
packages are analyzed to insure that product labels, lot numbers, "freshness" dates, and the 
like, are properly positioned and legible. 

In many machine vision applications, it is essential that an object whose image is 
to be analyzed include a . calibration target. Often a cross-shaped symbol, the target 
facilitates determining the orientation and position of the object with respect to other 
features in the image. It also facilitates correlating coordinate positions in the image with 
those in the "real world," e.g., coordinate positions of a motion stage or conveyor belt on 
which the object is placed. A calibration target can also be used to facilitate determining 
the position and orientation of the camera with respect to the real world, as well as to 
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facilitate determining the camera and lens parameters such as pixel size and lens 
distortion. 



In addition to cross-shaped marks, the prior art suggests the use of arrays of dots, 
5 bulls-eyes of concentric circles, and parallel stripes as calibration targets. Many of these 
targets have characteristics that make difficult finding their centers and orientations. This 
typically results from lack of clarity when the targets and, particularly, their borders are 
imaged. It also results from discrepancies in conventional machine vision techniques used 
to analyze such images. For example, the edges of a cross-shaped target may be 
10 imprecisely defined in an image, leading a machine vision analysis system to wrongly 

interpret the location of those edges and, hence, to misjudge the mark's center by a fraction 
of a pixel or more. By way of further example, a localized defect in a camera lens may 
cause a circular calibration mark to appear as an oval, thereby, causing the system to 
misjudge the image's true aspect ratio. 

15 

In addition to the foregoing, many of the prior art calibration targets are useful 
only at a limited range of magnifications. Parallel stripes, for example, do not provide 
sufficient calibration information unless many of them appear in an image. To 
accommodate this, a machine vision system must utilize lower magnification. However, 
20 as the magnification decreases, so does the ability of the machine vision equipment to 

distinguish between individual stripes. Similar drawbacks limit the usefulness of the other 
prior art calibration targets for use in all but a narrow range of magnifications. 

Though the art suggests the use of checkerboard patterns as alignment marks, the 
25 manner in which images of those marks are analyzed by conventional machine systems 
also limits their utility to a limited range of magnifications. Particularly, prior art systems 
obtain alignment information from checkerboard marks by identifying and checking their 
corners, e.g., the eight black (or white) comers in a black-and-white image. By relying on 
corners, the systems necessitate that images show entire checkerboards, yet, with 
30 sufficient resolution to insure accurate detection and analysis. 
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An object of this invention is to provide an improved calibration targets ana 
methods for machine vision analysis thereof. 

A related object is to provide calibration targets and analysis methods reliable at a 
wide range of magnifications. 

A further object is to provide such methods as can be readily implemented on 
conventional digital data processors or other conventional machine vision analysis 
equipment. 

Yet still another object of the invention is to provide such methods that can rapidly 
analyze images of calibration target without undue consumption of resources. 
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Summary of the Invention 

The foregoing objects are among those attained by the invention, which provides 
in one aspect a machine vision method for analysis of a calibration target of the type 
5 having two or more regions, each having a different "imageable characteristic" (e.g., a 
different color, contrast, or brightness) from its neighboring region(s). Each region has at 
least two edges — referred to as "adjoining edges" ~ that are linear and that are directed 
toward and, optionally meet at, a reference point (e.g., the center of the target or some 
other location of interest). The method includes generating an image of the target, 
10 identifying in the image features corresponding to the adjoining edges, and determining 
the orientation and/or position of the target from those edges. 

In another aspect, the invention provides a method as described above for 
analyzing a target of the type that includes four regions, where the adjoining edges of 
15 each region are perpendicular to one another, and in which each region in the target has a 
different imageable characteristic from its edge- wise neighbor. The edges of those regions 
can meet, for example, at the center of the target, as in the case of a four-square 
checkerboard. 

20 In yet another aspect, the invention provides a method as described above for 

determining an orientation of the target as a function of the angle of the edges identified in 
the image and for determining the location of the reference point as an intersection of lines 
fitted to those edges. In regard to the former, the invention provides a method of 
determining the orientation of a target in an image by applying a Sobel edge tool to the 

25 image to generate a Sobel angle image, and by generating a angle histogram from that 
angle image. In an alternate embodiment, the orientation is determined by applying a 
Hough line tool to the image and determining the predominant angle of the edges 
identified by that tool. 

30 In regard to the location of the reference point, one aspect of the invention calls for 

locating the adjoining edges by applying a caliper vision tool to the image, beginning at an 
approximate location of the reference point. That approximate location of the reference 
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point can itself be determined by applying a Hough line vision tool to the image in order 
to find lines approximating the adjoining edges and by determining an intersection of 
those lines. Alternatively, the approximate location of the reference point can be 
determined by performing a binary or grey scale correlation to find where a template 
5 representing the edges most closely matches the image. 

In another alternate embodiment, the approximate location of the reference point is 
determined by applying a projection vision tool to the image along each of the axes with 
which the adjoining edges align. A first difference operator vision tool and a peak 
10 detector vision tool are applied to the output of the projection tool (i.e., to the projection) 
in order to find the approximate location of the edges. 

The invention has wide application in industry and research applications. It 
facilitates the calibration of images by permitting accurate determination of target location 
15 and orientation, regardless of magnification. Thus, for example, an object bearing a target 
can be imaged by multiple cameras during the assembly process, with accurate 
determinations of location and orientation made from each such image. 

These and other aspects of the invention are evident in the drawings and in the 
20 description that follows. 
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A more complete understanding of the invention may be attained by reference to 
the drawings, in which: 

5 

Figure 1 A - 1C depict calibration targets according to the invention; 

Figure ID depicts the effect of rotation on the target depicted Figure IB; 

10 Figure 2 depicts an object according to the invention incorporating a calibration 

target of the type depicted in Figure IB; 

Figure 3 depicts a machine vision system according to the invention for 
determining the reference point and orientation of a calibration target; 

IS 

Figures 4 and 5 depict a method according to the invention for interpreting an 
image of a calibration target to determine a reference point and orientation thereof; and 

Figure 6 illustrates the magnification invariance of a target according to the 
20 invention. 
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Detailed Description of the Illustrated Embodiment 

Figures 1A - 1C depict calibration targets according to the invention. Referring to 
Figure 1 A, there is shown a target 10 according to the invention having three regions 12, 

5 14, 16. Each region is bounded by at least two linear edges that are oriented toward a 
reference location or reference point 18 on the target. Thus, for example, region 12 is 
bounded by edges 20, 24; region 14 is bounded by edges 20, 22; and region 16 is bounded 
by edges 22, 24. As evident in the drawings, the edges are shared by adjoining regions 
and, hence, are referred to below as "adjoining edges." Thus, region 12 shares edge 20 

10 with region 14; region 14 shares edge 22 with region 16; and region 16 shares edge 24 
with region 12. In the illustration, the reference point 14 is at the center of target 10, 
though, those skilled in the art will appreciate that the reference point can be positioned 
elsewhere. 

15 Each of the regions has a different imageable characteristic from its neighboring 

regions. As used herein, an "imageable characteristic" is a characteristic of a region as 
imaged by a machine vision system (e.g., of the type shown in Figure 3) and, particularly, 
as imaged by an image capture device used by such a system. For example, in the 
illustration, region 12 has the characteristic of being colored black; region 14, white; and 

20 region 1 6, gray. In addition to color, imageable characteristics useful with conventional 
machine vision systems ~ which typically utilize image capture devices operational in 
visual spectrum » include contrast, brightness, and stippling. 

Those skilled in the art will appreciate that any other characteristics by which a 
25 region may be identified and distinguished in an image are suitable for practice of the 
invention. Thus, for example, for a machine vision system that utilizes a temperature- 
sensitive (or infrared) image capture device, an imageable characteristic is temperature. 
By way of further example, for a machine vision system that utilizes a nuclear decay 
radiation-sensitive image capture device, an imageable characteristic is emitted radiation 
30 intensity or frequency. 
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As shown in the illustration, the adjoining edges 20, 22, 24 comprise straight linear 
segments. Those edges are implicitly defined as the borders between regions that, 
themselves, have different imageable characteristics. Thus, for example, edge 20 is a 
straight linear segment defined by the border between black region 12 and white region 
5 14. Likewise, edge 24 is defined by the border between black region 12 and gray region 
16. Further, edge 22 is defined by the border between white region 14 and grey region 16. 

Figure IB depicts a calibration target 30 according to the invention having four 
rectangular (and, more particularly, square) regions 32, 34, 36, 38. As above, each region 

10 is bounded by at least two linear edges that are oriented toward a reference point 40 at the 
center of the target. Thus, for example, region 32 is bounded by edges 42, 44; region 34 is 
bounded by edges 42, 46; and so forth. As above, these edges are shared by adjoining 
regions. Thus, region 32 shares edge 42 with region 34, and so forth. Each region in 
target 30 has a different imageable characteristic from its edge- wise neighbor. Hence, 

15 regions 32 and 36 are white, while their edge-wise adjoining neighbors 34, 38 are black. 

Figure 1C depicts a calibration target 50 according to the invention having five 
regions 52, 54, 56, 58, 60, each having two linear edges directed toward a reference point 
62. The adjoining regions are of differing contrast, thereby, defining edges at their 
20 common borders, as illustrated. Although the edges separating the regions 52 - 60 of 

target 50 are directed toward the reference point 62, they do not meet at that location. As 
evident in Figure 1C, no marker or other element imageable characteristic is provided at 
reference point 62. 

25 Those skilled in the art will appreciate that, in addition to the calibration targets 

shown in Figures 1 A - 1C, targets with still more regions (or as few as two regions) and 
shapes, otherwise in accord with the teachings hereof, fall within the scope of the 
invention. Moreover, it will be appreciated that targets may be of any size and that their 
regions need not be of uniform size. Still further, it will be appreciated that the outer 

30 borders of the targets need not be linear and may, indeed, take on any shape. 
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Figure 2 depicts an object according to the invention for use in machine vision 
imaging, detection, and/or manipulation having a calibration target according to the 
invention coupled thereto. In the illustration, the object is an integrated circuit chip 70 
having coupled to the casing thereof a calibration target 72 of the type shown in Figure 

5 IB. Other targets according to the invention, of course, can likewise be coupled to the 
object 70. The targets can be coupled to the object by any known means. For example, 
they can be molded onto, etched into, or printed on the surface of the object. By way of 
further example, decals embodying the targets can be glued, screwed or otherwise affixed 
to the object. Moreover, by way of still further example, calibration plates incorporating 

10 the targets can be placed on the object and held in place by friction. In addition to 

integrated circuit chips, the object can include any other objects to which a target can be 
coupled, such as printed circuit boards, electrical components, mechanical parts, 
containers, bottles, automotive parts, paper goods, etc. 

15 Figure 3 depicts a machine vision system 80 according to the invention for 

determining the reference point and orientation of an object 82 having coupled thereto a 
calibration target 84 according to the invention and, particularly, a four-region target of 
the type shown in Figure IB. The system 80 includes an image capture device 86 that 
generates an image of a scene including object 82. Although the device may be 

20 responsive to the visual spectrum, e.g., a conventional video camera or scanner, it may 
also be responsive to emissions (or reflections) in other spectra, e.g., infrared, gamma-ray, 
etc. Digital image data (or pixels) generated by the capturing device 86 represent, in the 
conventional manner, the image intensity (e.g., contrast, color, brightness) of each point in 
the field of view of the capturing device. 

25 

That digital image data is transmitted from capturing device 86 via a 
communications path 88 to an image analysis system 90. This can be a conventional 
digital data processor, or a vision processing system of the type commercially available 
from the assignee hereof, Cognex Corporation, as programmed in accord with the 
30 teachings hereof to determine the reference point and orientation of a target image. The 
image analysis system 90 may have one or more central processing units 92, main memory 
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94, input-output system 96, and disk drive (or other mass storage device) 98, all ot the 
conventional type. 



The system 90 and, more particularly, central processing unit 92, is configured by 
5 prograrnming instructions according to teachings hereof for operation as illustrated in 
Figure 4 and described below. Those skilled in the art will appreciate that, in addition to 
implementation on a programmable digital data processor, the methods and apparatus 
taught herein can be implemented in special purpose hardware. 

10 Referring to Figure 4 there is shown a machine methodology according to the 

invention for interpreting an image of a target 84 to determine its reference point and 
orientation. The discussion that follows is particularly directed to identifying a four- 
region target of the type shown in Figure IB. Those skilled in the art will appreciate that 
these teachings can be readily applied to finding targets according to the invention, as well 

15 as to other targets having detectable linear edges that are oriented toward a reference 
location or reference point on the target, e.g., a prior art cross-shaped target. For 
convenience, in the discussion that follows, such linear edges are referred to as "adjoining 
edges," regardless of whether they are from calibration targets according to the invention 
or from prior art calibration targets. 

20 

In step 100, an image of the target 84 (or of the target 84 and object 82) is 
generated, e.g., using image capture device 86, and input for machine vision analysis as 
discussed below. The image can be generated real time, retrieved from a storage device 
(such as storage device 98), or received from any other source. 

25 

In steps 102 - 108, the method estimates the orientation of the target in the image 
using any of many alternative strategies. For example, as shown in step 102, the method 
determines the orientation by applying a conventional Hough line vision tool that finds the 
angle of edges discernable in the image. In instances where the target occupies the entire 
30 image, those lines will necessarily correspond to the adjoining edges. Where, on the other 
hand, the target occupies only a portion of the image, extraneous edges (e.g., from other 
targets) may be evident in the output of that tool. Although those extraneous edges can 
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generally be ignored, in instances where they skew the results, the image can be 
windowed so that the Hough vision tool is only applied to that portion that contains the 
target. Once the angles of the lines has been determined by the Hough line tool, the 
orientation of the image is determined from the predominant ones of those angles. 
5 Alternatively, the angle of the image can be determined by taking a histogram of the 
angles. 

The Hough vision tool used in step 102 may be of the conventional type known 
and commercially available for finding the angle of lines in image. A preferred such tool 
10 is the Cognex Line Finder, commercially available from the Assignee hereof, Cognex 
Corporation. 

An alternative to using a Hough vision tool is shown in step 106. There, the 
illustrated method determines the orientation of the target by applying a Sobel edge tool to 
15 the image to find the adjoining edges. Particularly, that tool generates a Sobel angle 
image that reveals the direction of edges in the image. As above, where the target 
occupies the entire image, the adjoining edges will be the only ones discerned by the 
Sobel edge tool image. Where, on the other hand, the target occupies only a portion of the 
image, any extraneous edges can be ignored or windowed out. 

20 

The Sobel edge tool may be of the conventional type known and commercially 
available for finding lines in image. A preferred such tool is the Cognex Edge Detection 
tool, commercially available from the Assignee hereof, Cognex Corporation. 

25 Once the Sobel angle image is generated, in step 106, the orientation of the target 

in the image is determined by generating a histogram of the edge angle information; see, 
step 108. From that histogram, the target orientation can be determined by taking a one- 
dimensional correlation of that histogram with respect to a template histogram of a target 
oriented at 0°. Where a Sobel magnitude image is generated, in addition to the Sobel 

30 angle image, such a histogram can be generated by counting the number of edges greater 
then a threshold length at each orientation. 
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As a still further alternative to applying a Hough vision tool or Sobel edge tool, the 
method contemplates obtaining the angle of orientation of the target from the user (or 
operator). To this end, the user may enter angle orientation information via a keyboard or 
other input device coupled with digital data processor 90. 

5 

In steps 1 10 - 1 18, the method determines the location, i.e., coordinates, of the 
target reference point in the image. Particularly, in step 1 10, the method can apply a 
Hough vision tool, as described above, to find the angle of lines discernable in the image. 
A conventional Hough vision tool determines, in addition to the angle of lines in an image, 
10 the distance of each line, e.g., from a central pixel. As above, where the target occupies 
the entire image, those lines will be the only ones discernable by the Sobel edge tool 
image. Where, on the other hand, the target occupies only a portion of the image, any 
extraneous edges can be ignored or windowed out. 

15 As above, the Hough vision tool used in step 104 may be of the conventional type 

known and commercially available for finding the angle and position of lines in image. 
Once again, a preferred such tool is the Cognex Line Finder, commercially available from 
the Assignee hereof, Cognex Corporation. Those skilled in the art will, of course, 
appreciate that steps 102 and 1 10 can be combined, such that a single application of the 

20 Hough vision tool provides sufficient information from which to determine both the 
orientation of the target in the image and its reference point. 

As an alternative to using a Hough vision tool, the method can apply a projection 
vision tool to the image in order to find the position of the lines discernable in the image; 

25 see, step 1 12. The projection tool, which maps the two-dimensional image of the target 
into a one-dimensional image, is applied along the axes defined by the edges in the image. 
As those skilled in the art will appreciate, the location of the edges can be discerned from 
by finding the peaks in the first derivatives of each of those projections. As above, where 
the target occupies the entire image, those lines will be the only lines discernable by the 

30 Sobel edge tool image. Where, on the other hand, the target occupies only a portion of the 
image, any extraneous edges can be ignored or windowed out. 
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The projection vision tool used in step 1 12 may be of the conventional type known 
and commercially available for mapping a two-dimensional image of the target into a one- 
dimensional image. A preferred such tool is that provided with the Cognex Caliper tool 
commercially available from the Assignee hereof, Cognex Corporation. 

In step 1 14, the method uses the information generated in steps 110 and 1 12 to 
compute the location of the reference point, particularly, as the intersection of the lines 
found in those steps 110 and 112. 

As an alternative to using the Hough vision tool and the projection tool, the 
method can apply determine the location of the reference point by performing a binary or 
grey scale correlation on the image; see step 116. To this end, the method uses, as a 
template, a pattern matching the expected arrangement of the sought-after edges, to wit, a 
cross-shaped pattern in the case of a target of the type shown in Figure IB. The use of 
correlation vision tools for this purpose is well known in the art. The template for such an 
operation is preferably generated artificially, although it can be generated from prior 
images of similar targets. 

As still another alternate to the Hough vision tool and the projection tool, the 
method can apply a grey-scale image registration using the sum of absolute differences 
metric between the image and a template; see step 118. To this end. the method uses, as a 
template, a pattern matching the expected arrangement of the sought-after edges, to wit, a 
cross-shaped pattern in the case of a target of the type shown in Figure IB. The template 
for such an operation is preferably generated artificially, although it can be generated from 
prior images of similar targets. A preferred grey-scale image registration tool is disclosed 
in United States Patent No. 5,548,326, the teachings of which are incorporated herein by 
reference. 

Although steps 110 - 118 can be used to determine the approximate location of the 
reference point of the target in the image, the method utilizes optional steps 120 and 122 
to refine that estimate. These two steps are invoked one or more times (if at all) in order 
make that refinement. 

13 
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In step 120, the method applies a conventional caliper vision tool to tmd points in 
the image that define the adjoining edges of the regions. On the first invocation of step 
120, the method applies calipers along fifty points along each edge (though those skilled 
in the art will appreciate that other numbers of points can be used), beginning with points 
5 closest to the estimate of the reference point, as discerned in steps 110-118. The calipers 
are preferably applied a small distance away from the actual estimate of the reference 
point to avoid skewing the analysis due to possible misprinting of the target at that point, a 
missing pattern at that point (e.g., Figure 1C), or a too-high spatial frequency at that point 
(e.g., Figure IB). In step 120, the method then fits a line to the points found along each 
10 edge by the caliper tool, preferably, using a conventional least squares technique. 

In step 122, the method computes a refined location of the reference point as the 
intersection of the lines identified in step 120. Although conventional geometric 
calculations can be performed for this purpose, preferably, the reference point is computed 

15 using conventional least squares techniques. Preferably, when examining the image of a 
symmetric calibration target of the type shown in Figure IB, the method utilizes the same 
number of points on either side of (and closest to) the reference point for purposes of 
fitting each line. This minimizes the bias otherwise introduced by a conventional edge 
detection technique in finding edges that are defined only by dark-to-light (or light-to- 

20 dark) transitions. 



Calibration targets of the type shown in Figure IB are advantageously processed 
by a method according to the invention insofar as they further minimize bias otherwise 
introduced by a conventional edge detection techniques. In this regard, it will be 
appreciated that such bias is reduced by the fact that "opposing" adjoining edges (i.e., 
edges that oppose one another across the reference point) define straight linear segments 
that change polarity across the reference point. That is, those segments are defined by 
regions that transition ~ preferably, equally in magnitude - from light-to-dark one one 
side of the reference point, and from dark-to-light on the other side. This is true for all 
"symmetric" calibration targets according to the invention, i.e., targets in which opposing 
edges define straight linear segments that are opposite polarity on either side of the 
reference point. 
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On the second and subsequent invocations of step 120, the method specifically 
applies the caliper tool at points along the lines found in the previous iteration. 
Preferably, these points are at every pixel in the image that lies along the line, though, 
those skilled in the art will appreciate that few points can be used. The calipers can be 
5 applied orthogonal to the lines, though, in the preferred embodiment, they are applied 
along the grid defined by the image pixels. The caliper range decreases with each 
subsequent invocation of step 120. The method continues applying the calipers, beginning 
at the estimated center point until one of the four following situations occurs: no edge is 
found by the caliper applied at the sample point; more than one edge is found by the 

10 caliper and the highest scoring edge is less than twice the score of the second highest 
scoring edge (this 2X comes from the CONFUSION_THRESHOLD); the distance 
between a computed edge point and the nominal line (computed from the previous 
invocation of step 120) is larger than a threshold (which threshold decreases with each 
subsequent invocation of step 120); or, the caliper extends outside of the image. As 

15 above, in step 120, the method then fits a line to the points found along each edge by the 
caliper tool, preferably, using a conventional least squares technique. 

In the second and subsequent invocations of step 122, the method computes a 
refined location of the reference point as the intersection of the lines identified in step 120. 
20 As above, although conventional geometric calculations can be performed for this 
purpose > preferably, the reference point is computed using conventional least squares 
techniques. 

Referring to Figure 5, there is also shown a machine methodology according to the 
25 invention for interpreting an image of a target 84 to determine its reference point and 

orientation. Particularly, in step 150, the method calls for generating an image of a target 
84 and, particularly, of a target according to the invention having two or more regions, 
each region being defined by at least two linear edges that are directed toward a reference 
point, and having at least one of the regions having a different imageable characteristic 
30 from an adjacent region. Step 1 52 can be effected in the manner described in connection 
with step 100 of Figure 4, or equivalents thereof. 
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In step 152, the method analyzes the image to generate an estimate of an 
orientation of the target in the image. Step 152 can be effected in the manner described in 
connection with steps 102- 108 of Figure 4, or equivalents thereof 

5 In step 154, the method analyzes the image to generate estimate of a location of the 

target's reference point. Step 154 can be effected in the manner described in connection 
with steps 1 10 - 118 of Figure 4, or equivalents thereof. 



In step 156, the method analyzes the image to refine its estimates of the location of 
10 the reference point in the image and the orientation of the target in the image. Step 156 
can be effected in the manner described in connection with steps 120 - 122 of Figure 4, or 
equivalents thereof. 

Calibration target and methods for analysis according to the invention are 
15 advantageous over prior art targets and methods insofar as they are magnification 

invariant. By analyzing the adjoining edges of targets, methods according to the invention 
insure reliance on features (to wit, regions) that retain the same imageable appearance 
regardless of magnification. This is in contrast to prior art targets and methods, which rely 
on individual lines (or dots) to define calibrating features. As noted above, the imaging 
20 appearances of such lines and dots change with varying degrees of magnification. Even 
the prior art methods that analyze checkerboard targets rely on analysis of corners, which 
are not magnification invariant. 



The magnification invariance of targets and methods according to the present 
25 invention is illustrated in Figures 6A - 6C. Referring to Figure 6A, there is shown an 
imaging setup wherein camera 200 images a target 202 (on object 204) from a height x. 
An image generated by camera 200 is displayed on monitor 206 of workstation 208. 
Referring to Figure 6B, there is shown an identical imaging setup, except insofar as a 
camera (of identical magnification) images a target (of identical size) from a greater 
30 height, x\ Likewise, Figure 6C shows an identical imaging setup, except insofar as a 

camera (again, of identical magnification) images a target (again, of identical size) from a 
still greater height, x". Comparing monitor images depicted in Figures 6A - 6C, it is seen 
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that the images of the targets, generated by the cameras and displayed on the monitors, are 
identical in appearance regardless of the relative heights (x, x\ and x") of the camera. This 
magnfication invariance results from the fact that the target has the same appearance 
regardless of height (or equivalently, magnification), within specified limits thereof. 
Those skilled in the art will appreciate that those limits are greater than the analagous 
limits for prior art targets (e.g., cross hairs, parallel stripes, etc.). 



Still another advantage of calibration targets and methods according to the 
invention is that they permits angular orientation to be determined throughout a full 360° 
10 range. With reference to Figure ID, for example, the relative positions of the regions can 
be used to determine the overall orientation of the target, i.e., whether it is rotated 0°, 90°, 
180°, or 270°. This information can be combined with a determination of relative 
orientation made by analysis of the adjoining edges as discussed above to determine the 
precise position of the target. 
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APPENDIX I 



Patent Application for 



MACHINE VISION CALIBRATION TARGETS AND METHODS 
OF DETERMINING THEIR LOCATION AND ORIENTATION 



Vision Tool Description 



THE FOLLOWING APPENDIX IS NOT BELIEVED TO BE NECESSARY FOR 
ENABLEMENT OR BEST MODE DISCLOSURE OF THE INVENTION DISCLOSED AND 
CLAIMED IN THE ACCOMPANYING APPLICATION. 
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Edge Detection Tool 

This chapter describes the Edge Detection tool, which includes edge detection ana peak 
detection. 

Edge detection takes an input Image and produces two output images: an image of the 
edge magnitude of eacn input pixel and an image of the edge angle of each input pixel. 
When combined with other Cognex software toots, the information produced by edge 
detection can be used to locate objects within an image. Edge pixel information can be 
used to detect the rotation of an object or detects such as scratches, cracks, or particles. 
You can aiso gauge now snarpty an image is focusea with edge pixel information. 

Peak detection is useful in any application m which knowledge of the local maximum valuee 
in a two-dimensional space is useful. Peak del ecu on takes an input image and produces 
an output Image containing only those pixels tn the input image with higher values than 
neighboring pixels. A typical input image to peek detection Is an edge magnitude image; 
the output Image contains only the highest magnitude edge pixels. The edge detection 
function can optionally use peak detection to postprocess the edge detection results so 
that only the strongest eages remain. 

This cnapter has nine sections as follows: 

An Overview of Edge Detection describes the goals of edge detection, defines an edge 
pixel, and descnbes now the Edge Detection tool finds edge pixels. This section also 
explains the two properties of a pixel that are calculated by the Edge Detection tool, edge 
magnitude and angle, and ends with a sample application. 

Using Edge Detection describes, in general terms, the interface to the Edge Detection tool. 
ttus section contains a atscussion of tne compression tables mat affect the amount of 
memory used by the edge detector. 

Edge Detection Enumerations and Data Structures describes the data structures and 
enumerations that the edge detection functions use. Types and data structures that 
support peak detection are discussed in Peak Detection Enumeration Types and Data 
Stn/ctures below. 
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Edga Defection Functions describes tne functions mat implement edge detection. 

Maxmizing the Performance of VC2 Vision Coprocessor lists the edge detection parameter 
settings thai you should avoia if you want to maximize th« oerformance ot the VC2 vision 
coprocessor during edge detection. 

Overview ot Peak Detection defines peaK pixel and gives examples ot some applications 
of psak detection. 

Using Peak Detection describes me interface to peak detection. 

Peak Detecvon Enumeration Types and Data Structures describes trie peak detection 
enumer&Dons end oata structures. 

Peak Detecvon Functions describes the peak detection functions. 
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Some Useful Definitions 



edge pixel 

edge detect©/ 
compraaaton table e 



Sobs I operators 

horlzonUi edge 
component 

vertical edge component 
edge magnitude 
edge angle 



peak pixel 

peak detection 

6-polnt neighborhood 

4-polnt neighborhood 
2 -point neighborhood 



Pixel that has a differ ent value from one or more of its 
eigni netgnoorwg pixels. 

Operator that finds edge pixels in on imago. 

Two tables that are created during edge detection 
initialize con: the edge compression table and the 
magnitude compression table. These tables ere used by 
the Edge Detection tool to meD data into a smaller range 
of values. 

Two 3x3 operators that the Edge Detection tool uses to 
locate edge pixels in an image. 

Value returned by the horizontal Sob el operator when it is 
applied to a pixel in an image. 

Value returned by the vertical Sobel operator when it ts 
applied to a pixel in an image. 

Value mat increases as the difference in grey levels 
between neighboring pixels increases. 

Orientation of an edge, with respect to the x-axis of the 
image, at an edge pixel. The Edge Detection tool defines 
me angle ol a pixel as arctan(v/h), where v is the vertical 
eoge component and h is the horizontal edge component 
of the pixel. 

Pixel with e value greater than or equal to some or all of 
its neighboring pixels' values. 

Finding some or all peak pixels in an image. 

Pixels (hat are horizontally, vertically, and diagonally 
adjacent to a pixel. 

Pixels that arc horizontally and vertically adjacent to a 
pixel. 

Neighbors of a pixel along e single axis. Peak detection 
supports a horizontal and a vertical 2-point neighborhood 
operator. 
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symmetric peak 
••ymmotrte peak 

alhaxea paak 
tlngta-axle peak 
plateau 



Peak whose pixel value is greater than or equaJ to me 
values of each of its neighbors. 

Peak wnose pixel value is greater than the values of its left 
and tower neighbors and greater than or equal to the 
values of Its right and upper neignbors. 

Pixel that satisfies the peak definition (symmetric or 
asymmetric) along all axes in Its neighborhood. 

Pixel that satisfies the peak definition (symmetric or 
aaymmetno) along at least one axis m its neighborhood. 

In Deak detection, a reoion of contiguous oixels of the 
same value. 
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An Overview of Edge Detection 

The Edge Detection Tool finds edge pixels in an image. This overview of edge detection 
contains the following doccrtDtionr 

• Definition of the problem of edge detection 

• Overview of tne tasks that the Edge Detection tool performs 

• Definition of edge pixel 

• Description ol the Sob el oo orator, which is used by (he Edge Detection toot to locale 
edge pixels 

Definitions of the angle and magnitude of an 0090 pixel, and deeerlptrone of the two 
images produced by the Edge Detection toot: the magnitude image and the angle 
image 

• Description of edge detection preprocessing 

• Description of edge detection postprocessing 

• Description of a sample application that uses both the magnitude and the angle 
images to create an angle "signature* of a shape 

Edge Detection 

The Edge Detection tool locates edges and determines each edge's angle and magnitude. 

An edge occurs wherever adjacent areas of an image have different grey values. Using this 
definition the triangle >" Figure 66 has a single edge. 




Figure 66. A figure with an edge 
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The definition of edge can be further refined as the a reeled border between grey areas at 
an image, where Erection is defined as a vector normal to the edge. Using this defirition. 
the triangle in Figure 67 has throe directional edges, represented by the three vectors. The 
angle or the edge is me counterclockwise angle of the vector normal to the eoge, with 
resDect to the horizontal axis of me image. See the section Angle image on Dage 1 Si for a 
discussion of edge angle. 




Hguro o?. Vectors indicating The oage angles of a triangle 

Along with an angle, an edge has a magnitude, which reflects the amount of tha difference 
between grey levels on either side of the edge. As the difference in grey levels increases, 
the magnitude increases. See the section The Magnitude Image on page 176 for a 
discussion of edge magnitude. 

The triangle in Figure 68 has the same three edges as the triangle in Figure 67: however, 
these edges are ot lower magnitude. This lower magnitude is Illustrated by the shortened 
lengrn of ins tnree Direction vectors. 




Figure 68. A triangle with low-magnitude edges 
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Edges are actually located pixel by pixel, figure 69 contains a magnified view of a portion 
of an edge in an Image. Each cell in the figure represents The grey level of a single pixel. 
Notice that, on this "microscopic" level, there are many edges between Dixels in which the 
grey levels on either side of the edge differ by a few percentage points. 
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Figure BO. Grey levels in a magnified portion of en edge 

The Edge Detection tool locates edges in an image by identifying each pixel that has a 
different value from one of its neignboring pixels. It calculates the angle ano magnitude of 
eacn edge pixel. It also oroudes a means of classifying edges, so that low-magnitude 
edges are not reported. 
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Edge Pixels 



A pixel is en &dg» pur*/ if it hae a different value from at least one of ite eight neighboring 
pixet9 and is not on the border of an image. In Figure 70. the four snaded pixels are edge 
pixels. 
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Figure 70. The shaded pixels are edge pixels 

Figure 71 contains a grid representing grey levels in an image with the highest magnitude 
edge pixels shaded. Notice that border pixels (pixels along the edge of the image) are not 
edge pixels. 



0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 




3 


0 


0 


0 


0 


0 


0 


0 






4 


4 






0 


0 


0 


0 


0 




4 


4 


4 


4 






0 


0 


0 






4 


4 


4 


4 


4 


4 


0 






4 


4 


4 


4 


4- 


4 


4 


4 


3 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 


4 



Figure 71. Highest magnitude edge pixels in en image 
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The Edge Detection Operators 

The Edge Detection tool finds edge pixels using the Sob*/ 3x3 neighborhood operators. 
During edge detection, the Sobel operators are applied to each pixel In an input image. 
This ooeraton produces two values for each pixel. One value represents the vortical edge 
component 1 or the pixel; the other value represents the horizontal edga component lor the 
pixel. These two values are then used to compute the edga magnitude and edge angia of 
the pixel. 

Sobel edge detection uses two 3x3 neighborhood operators to locate edge plxeb In an 
image. The horizontal Sobel operator detects the horizontal (x> edge component for a pixel. 
The vertical Sooei operator detects me vertical (y) eoge component tor a pixel. Ftgure 72 
snows the Sobel operators. 
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Horizontal operator Vertical operator 

Figure 72. Sobei operators 

The Sobel operator works only on pixels with eight neighboring pixels. When the Sobel 
operator is applied to a pixel on the border of an image, the result is defined to be 0 
because there are not ervougn neighboring pixels to calculate an edge value. 

The horizontal and vertical Sobel operators are 3x3 linear operators. These operators 
compute a value for a pixel (x.y) in the following way: 

• The grey level of tne pixel (x.y) is multiplied by the value in the center of the 3x3 bnear 
operator. 

• The grey level of the neighboring pixel (x-1,y~l) is multiplied by the first value in the 
top row of the 3x3 linear operator. 

• The grey level of the neighboring pixel (x.y-1) is multiplied by the second value in the 
top row of the 3x3 linear operator. 

• This continues untd tne product of eeoh peir of values Is calculated. 

• These products are then summed to produce the output. For the Sobel operator the 
value ts interpreted as horizontal or vertical edge component or tne pixel. 
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If the result of applying at least one of the Sob el operators to e pixel is nonzero, that pixel 
is an edge pixel. 
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Sobel operator Image pixels 



Figure 73. Applying the Sobel ooerator 

in Figure 73, the horizontal Sobel ooerator is applied to the shaded pixel in the image. The 
resultant edge pixel value is 6. which is calculated as follows: 

-M*0'UI-2 + -21t02t23t.M*03»13 = 6 

Atote:The range of pixel values in Figure 73 and in most examples in this document is much 
smaller than the typical range of pixel values in an image. This is done to simplify the 
examples. 

Figure 74 shows all the vertical and horizontal edge component valuec for a email image. 
The uoper grid is the image; the values represent the grey levels of each pixel. The gnd to 
the lower left of the image contains horizontal edge component values for tha imaga. Each 
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cell in the grid contains the result of applying the Sobel horizontal operator to the 
corresponding input Image pixel. The grid to the lower right contains the vertical eage 
components for the image. It is computed using the Sobel vertical operator. 
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Horizontal edge component values 



Vertical edge component values 



Figure 74. Horizontal and vertical edge component vaJues computed 
with me Sobel operators 

The horizontal and vertical edge component values computed with the Sobel operator are 
not returned by the edge detection functions. They are used to compute the edge 
magnitude ano eage angie or eacn pixel tn me image. 
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The Output Images 



Edge magnitudes are stored man outDut image that chowe me edge magnitude ot each 
corresponding pixel in the input image. Edge angles are stored in anotner output image 
thai shows the edge angle of each pixel. You can chocs© wvnicn outout tmagac to create, 
magnitude, angle, or both. 

These two images are described in this section. 

Tba Magnitude Imago 

Tne eage magnitude for a given pixel is a function of the horizontal and vertical edge 
components ot the pixel. If x is the horizontal edge component value and y is the vertical 
eage component value tor some pixel, men the eagemagnituoa. m. for that pixel is oennea 
as follows: 



This formula suggests a geometric interpretation of the data: if xls the horizontal edge 
component and y is me vertical edge component tne magnitude is a vector (rem ine origin 
of a two-dimensional coordinate system to me point (x.y). Figure 75 shows the geometric 
relationehip between the edge magnitude and the two edge oompononts. 



Now: For imptementation-soecitic reasons, the Edge Detection tool uses the formula 
described above to compute edge magnitude and men scales tne magnitude upward 
by approximately 16%. 




Vertical edge 
component 




Edge magnitude 



* Horizontal edge 
component 



Figure 76. Geometric representation ot edge magnitude 
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You can choose the numoer ot bits tnat can ba used 10 express magnitude. Because 
magnitude values, computed with vertical and horizontal edge component values, can 
exceed this number of bits, magnitude values are comDfessod to an integer value that can 
do expressed in the number of bits you have specified. You can control how magnitude 
values are compressed; you can choose a method suODlied by the Edge Detection too), 
such as a logarithmic or linear map, or you can supply a magnitude compression map of 
your own. For descriptions of the maps supplied by the Edge Detection toot and for 
information on using your own map. see the section Two Compression Tables on page 1 92. 

The sue and depth of the magnitude image ere the same as that of the input image. Since 
all border pixets have horizontal and vertical edge pixel values of 0. the border of the 
magnitude image contains all xeros. You can display en edge magnitude image: higher 
magnitude edges are brighter. 

Optionally, edge detection can return the numoer of edge pixels in the input image and the 
sum ot all edge magnitude values. By dividing the sum of all edge magnitude values by the 
size of the image, you can compute the average edge magnitude of an image. If you have 
two Images of the same scene, you can determine which image is in sharper focus by 
comparing average edge magnitudes of the two images: the image with the higher average 
magnitude is in sharper focus. 
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Figure 76 contains en image, the image's honzomat and vertical eaga components 
computed with Sobel operators, and the output image containing the edge magniiude 
values for each pixel. The edge pixels with the highest magnitude, reoresenttng the actual 
edge, are shaded. 
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Figure 76. Edge magnitude values computed with tne Sobei operators 



Angle rmage 



The edge detection tu notions produce an image containing tne angle of each edge pixel. 
If x Is the horizontal edge component value end y is the vertical edge component value, the 
edge angle id the counterclockwise angle between the horizontal axis end tne magnitude 
vector. 

Figure 77 shows tne geometric relationship between the edge angle, the edge magnitude, 
and the two edge components. In this figure. Mis the magnitude vector and e is the edge 
angle. 



Vertical edgo 
component 




Horizontal edge 
component 



Figure 77. Geometric interpretation of edge angie 

Edge angles are stored tn the angle image in binary form. They are represented as 
fixed-point numbers with tne decimal point to the left of tne most significant bit and are 
interpreted a9 fractions of 360°. You can choose the maximum number or pits avaiiaoie to 
express an angle: this value must be less than or equal to B. If n is the maximum number 
of bits used to express angle, and x is the binary representation of an angle, you can 
convert xto degree notation by using the following formula; 



angled) = 



360x 
2 n 



Note that with a 6-bit representation, angles can be expressed to the nearest 5.626*; with 
a 7-bit representation, an angle can be expressed to the nearest 2.8°: with an 8-bit 
representation, an angle can be expressed to the nearest 1 .4*. 
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The edge angle is computed using the arc tangom function if x is the horizontal edge 
component and y is the vertical edge component for a pixel, (he eage angle for that pixel 
re calouJatod as shown in Table 3. 
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If 


Cdg* Angi* 


x>0 


y > 0 


arctary y/x ) 


x > 0 


y < 0 


360-fircten( ry/xt ) 


x > 0 


y » 0 


0 


x < 0 


y > 0 


180- arctan( ry/xi ) 


j* < 0 


y < o 


1 80 * arc»n( y/x ) 


X < 0 


y-0 


160 


x = 0 


y > 0 


90 


XoO 


y < 0 


270 


x «0 


y * 0 


0 



Table 3. Formulae tor computing edge angles 
The computation is designed so that an edge angle e is in tne following range: 

0* s e < 360° 
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Figure 78 eomamssix rows. Each row contains (from left to right) an imago with the central 
pixel shaded, the vertical and horizontal edge component value of the shaded pixel, the 
image with a voc tor supor imposed snowing tn« *dg* angle of the contra) pucol. and thm 
formula used to compute me edge angle. Notice that the eauaoons defining the edge angle 
vaiu* are design ad so that the vector always points to the brighter side of the edge. 



00000 
□0000 
00 00 

DQODn 
DEIQQD 



Vertical edge = -4 
Horizontal edge » 0 



00000 
00000 
00 00 

nakiDo 
Donoa 



Angle ° arc tan(-4/0) = 270 # 



000QD 
000OO 

00 nn 
000nn 

000DD 



Vertical edge = 0 
Horizontal edge - 4 



000nn 
0Q0DD 
00 ^3D 
000DO 
000UO 



Angle = arc tan(074) - 0° 



00000 
EI0000 

on 00 
aaa00 

DDQD0 



Vertical edge - -3 
Horizontal edge - -3 



00000 
□0000 

on 00 

IIUDH0 

annti0 



Angle « arc tan<-3/3) «= 225* 



0onoa 

00OOQ 

00 nn 

0000H 
00000 



Vertical edge - 1 
Horizontal edge - i 



Tl 



0non 
00nnn 
00 nn 

Q000U 
00000 



Angle » arc tan( 1/1 ) - 45° 



nnnnn 
nnnnn 

00000 
00000 
00 00 



Vertical edge - a 
Horizontal edge = 0 



nann0 
nnn00 
nn 00 

U0000 
00000 



Veaical edge « 3 
Horizontal edge ° -3 



nnnnn 
nnnnn 

00000 
00000 
00 00 



nnnn0 
nrnH0 
nn 00 

U0000 
00000 



Angle - arc tan(4/0) = 90* 



Angle = arc tan(3/-3) » 135* 



Figure 78. Computing me angle of various edges 
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By setnng a flag in the edge detection parameters, you can specify that you want angles 
computed tn the range 0* to 1 80° instead of the default range, 0° to 360°. When you select 
mo smaller range, an edge angle n that ts greater than 1 80° maps to eoge angle (n — 1 80*). 
For instance. 300° maps to 120*, and 270° maps to 90°. 

Use tne reduced angle range when you want me same eoge angle results regardless of 
tne polarity of the image. As shown in Figure 79, when the range of angles is restricted to 
me range 0° to 180°. the same edge angles are computed tor a black object on a white 
background as lor a white object on a black background. When the range of angles is 0* 
to 360* (also Figure 79) this is not the case: reversing the polarity of the image reverses the 
edge pixel angle. 




Figure 79. Effect of reversed polarity on edge angles for the two ranges 

The binary representation ot an angle in tne range 0° to 1 80* is similar to the representation 
ot an angle in the range 0° to 360*. It n is the maximum number of bits used to express 
angle and jris the binary representation ot an angle in the range 0* to 180°, you can convert 
x to oogree notation by applying the following formula: 

180* 
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Edge 



Detection Preprocessing 



If you supply a pixel nrmp. the edge detection function usee it to preproeeaa your input 
imago before It calculates the horizontal and vertical edge components. For a description 
of Dixoi mapping, boa Cfmptor 3. Pixoi Mapping. 



Although edge angles are defined for all pixels in an image, they ere significant only at the 
pixels with the greatest edge magnitudes. There are many edges between pixels where the 
grey levels on either aide of the edge differ by only • few percent. Theee inatgnifioanx edge 
pixels, due to frame grabber noise end random fluctuations in pixel values, can clutter your 
edge megnitude image and make it difficult (o evaluate the data in that image. Atao edge 
magnitude values usually increase gradually in a "smear* as they approach an edge and 
then decreaee as the edge ie crossed. You might wont to c nor pan the magnitude image by 
filtering out some edge pixels that are near, but not at. an edge. 

The Edge Detection tool supplies three postprocessing methods. Two of these methods, 
setup and run-time thresholding, set the lowest edge magnitude pixels to 0. The third 
method, peak detection, takes an edge magnitude image ana processes n so mat. tor any 
small Pixel neighborhood, only the highest edge magnitude pixels remain and the other 
pixels are set to o. See tne section Pea* Detection on an Eage Magnitude tmage on 
page 229 for a complete discussion of this method. 

After postprocessing, edge angles corresponding to zero edge magnitude pixels are also 
set to 0 unless you have specified run-time thresholding but not peak detection. In this 
case, edge angles corresponding to zero edge magnitude pixels can be nonzero, but 
should be considered unaefined. 

Setup Threeholding 

You can supply a setup time rejection percentage. This is a value n that represents a 
percentage of the highest possible edge magnitude value. With this method, any 
magnitude values in the first n percent of all possible values are forced to 0. For example, 
if you provide 6 bits to express magnitude values and specify that the lowest 5 percent of 
magnitude values are forced to 0. then eny magnitude below 3 is forced to 0 since 5 
percent of 63 (truncated) Is 3. 

RurvTime Thresholding 

You can supply a run-ttme rejection percentage. Ths is a value n that represents e 
percentage of the highest actuai edge magnitude value in a magnitude image. With this 
method, magnitude values in the first n percent of tne magnitude values in a specific image 



Edge 



Detection Postprocessing 
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are forced to 0. For example, if the highest magnitude value in a particular image is Si . and 
if you specify thai the lowest S percent of tha run-time magnitude values ere forced to 0, 
then any maonitude below 2 is forced to 0. since 5 percent ot 51 (truncated) is 2. 

Using a setup rejection percentage is faster than using a run-time rejection percentage, 
since the values that are forced to 0 are computed once, before anymagnttuoe images are 
created. However, the run-time method is more flexible and robust. If you are producing 
magnitude images ot the same scene out under varying lighting conditions, you snouo use 
a run-time rejection percentage since the proportion of magnitude values forced to 0 
remains stable as tha range of magnitude values changes (due to varying illumination). 

Postprocessing with Peak Detection on a uagnmioa knags 

Peak detection is a method of filtering en image eo that only ooak ptxa/s remain and other 
pixels are set to 0. Peek pixels nave a value greater than or equal to some or aft of their 
neighboring pixels' value*. 

You can set a flag in the edge detection parameters structure that instructs the edge 
detection function to run peak detection on the magnitude Image during postprocessing. 
All nonpeak pixels are filtered out of the magnitude image and the edge angle image before 
they are returned. When you specify this, you must supply peak detection parameters in 
the edge detection parameter structure. 

The edge detection function does peak detection postprocessing after it has done setup 
end run-time thresholding. There will ba a 2-pixe4*wlda border of zoroe in a peatt-doleotod 
magnitude image. 

The section Peak Detection on an Edge Magnitude Image on page 229 discusses, in detail, 
the benefits of using peak detection on an edge magnitude image. This section also 
describes the proper way to parameterize peak detection so that it filters edge magnitude 
pixels correctly. However, you do not need a detailed knowledge of peak detection to use 
it with edge detection. Suggested peek detection parameters are supplied tn the section 
Data Structure cip_edge_params on pege 199. 

Figure 60 contains an image, its edge magnitude image, and the image that results from - 
peak detection. The edge magnitude image contains a smeared edge, which is sharpened 
by peak detection. 
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Figure 80. Peak detecting an edge magnitude Image to determine actual 
edge 
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Sample Application: Angle Histogram 

You can u— tne magnitude and angle imagoo of an objoot to areata an angle histogram of 
mat object. This ts a table that assigns a value n to each edge angle 6 it there are n edge 
pmalc (abovo a given magnitude threshold) whose edge angle is a. 

The angle histogram supplies a useful "signature* of an object's shape. You can use this 
signature to gauge similarity among objects, to detect rotations of an object, and to detect 
tlawa in an object. 

Once you have created edge magnitude and edge angle images for an object, you can 
create and display en angle histogram no follows: 

void angle_bist.ograjn ( res . chr eshold ) 

cip_«dge_reBule£ *r«s; /• result* structure pointer •/ 

int threshold; /* significant edge threshold •/ 

( 

int i,j,mag; 
int hisc(256] ; 

cu_clear(hist, s±xeo£ (hist) ) ; 

for U=0; l<res->mag_liag_p->width; i**) 

for (j=0; j<res->mag - ijng_p->height; j**) t 

BUig=reB->mag_U»g_p-»oat (res-— Btag_i-g_p , 1,3); 
if (nag > threshold) 

hist (res->ang_img_p->get lres->ang_i-,g_ p, i, j ))♦■»■: 

) 

cgr_hiitogr— >lct _imga.hict. 65) : 
caq_.di splay ( ) ; 
return ; 

) 
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Figure 81 contains four shapes; each shape is paired with its angle histogram. Note that 
tne square and the plus sign have identical angle histograms. The angle histogram of the 
circle is flat because there is no dominant angle along me edge of a circle. 
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Figure 87. Assortea snapes wnn rneir angle histograms 
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Figure 62 demonstrates tne effect of rotation on The angle histograms of a plus sign. The 
histogram shifts as the plus sign rotates. 
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Figure 82. Effect of rotation on sng/e histogram 



42 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 



PCT/US97/18268 



CaliperTool 4 



The Caliper Tool 

This section defines the Caliper Toot and describes in general terms how it works. The 
concepts introduced in this section are described in detail m tutor sections* 



The Purpose of the Caliper Tool 

The Caliper Tool is a too) for locating edges and edge pairs in an image. The edge of an 
object in an image is a change in grey value from dark to tight or light to dark. This change 
may span cevoral pixels. The Caliper Tool provided methods for ignoring edgea that are 
caused by noise or that are not of interest for a particular application. 

The Caliper Toot is modeled on the mechanical engineer's caliper, a precision device for 
measuring distances. You specify tne seperaDon between the caliper *iaws.* and the 
Caliper Toot searcnes an area you specify for edge pairs separated by that distance. You 
can aJso search for Individual edges when you know their approximate location in an 
Image. This is similar to using a caliper to measure depth. 

A typioal use for the Caliper Tool ia part inspection on an assembly lino. For example, in 
integrated circuit lead inspection, a part is moved in front of a camera to an approximately 
known location (the axpaciaa location). You use tne Caliper Tool to determine fre exact 
location of me left edge of the part by searching for a single light to dark edge at the 
expected location (see Figure 1 03). The edge that is closest to the expected location ia the 
left edge of tne part. 




figure 103. The Caliper Tool is used to locate thB iatt edge of the part. 
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One© you find the ten edge of the part, you can predict where the leads should tie within 
the image to specify a window in which to look lor leads (see Figure 104). You can then use 
the Caliper Toot to search for edge pairs consisting of 8 dark to light transit on followed bv 
a light to dark transition, and seperated by the ejected lead width. Wlm information 
calculated by the Caliper Tool, you can compare measured lead width and location to 
expected lead width and l oca Don to make an acceptfreject decision about the pan. 




Figure 104. The Caliper Toot can be used to measure lead width and 
load spacing on an integrated circuit. 



How the Caliper Tool Works 

The Caliper Tool uses projections map a two-oimensiona) window of an image (the caliper 
windoW) into a one-aimensional image. Projection collapses an image by summing the 
pixels in me airectton or me projection, wntcn tenas to amptiry eages in mat direction. 
Figure 105 shows an image, a caliper window, the one-directional image that results from 
projecbon. end a graph of the pixel grey values in the one-Oimensional image. 
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Image Caliper window 



1 

Proj action 

direction 




One-dimensional 
intensity image 



First edge Second edoa 

Figure 105. Overview of the Caliper Tool 

You may specify That the Caliper Tool perform pixel mapping before projection. Pixel 
mapping can be used to filler an image, attenuate or amplify a range of grey values, and 
otherwiao map pixel value©. Sao Chapter 3, Pixel Mapping, tn the Imago Processing 
manual for a complete description of pixel mapping. 

After projection, an edge filler is applied to me one-dimensional image to further enhance 
edge information and to smooth the image by eliminating minor grey value changes 
between neighboring pixels that are most likely caused by noise. The edge filter produces 



45 



SUBSTITUTE SHEET (RULE 26) 



98/18117 



PCT/US97/18268 



Caliper Tool 



a first derivative image (the mosi rapid changes in grey value in the projected image result 
in the highest peaks in the filtered image). Rgure 106 shows the one-dimensional image 
from Figure 105. an image generated by the edge niter, and a graph of thai imago. 




First edge Second edge 



Figure 706. An eage filter is applied to the one-atmensronm image that 
results from projection 

After me filter operation, edge detection is performed on the Altered image. Edge detection 
ic a method of evaluating peaks in the filtered imago and ignoring edgee that ara not of 
interest, such as eages representing noise in the image. Edge detection applies geometric 
constraints xnex you specify to each edoe or edge pair found in the image. Applying 
geometric constraints provides a way to limit tha number of edges ro evaluate by assigning 
a score to each edge or eage pair oased on such factors as the expected location of the 
edge or edge pair, the distance between edges in an edge pair, and the minimum grey 
level difference between pixels on each side of the edge. 

The Caliper Tool returns as many results as you have specified in the cctp_p«ram8 data 
structure and computes the geometric average of the score of eacn geometric constraint 
that you specify. For edges or edge pairs whose score equals or exceeds the accept 
threshold, tha Caliper Toot returns information such as the score end location of the edge 
or edge pair and the measured distance between edges in an edge pair. 

The controlling variables of an edge or edge pair search define a caliper, and include such 
information as the dimensions of the search window and the edge detection scoring 
parameters. The type of rne data structure mat defines a caliper is ectp.callper. 
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The Caliper Window and Projection 

This section describes how you specify me caliper window, which is the subset of an image 
in which to locate edges or edge pairs. It man describes orojacuon, which is tha mapping 
of the caliper window into a one-dimensional image. Final fy it describes the edge fitter. 
which enhances edges in the one-dimensional image. 

Caliper Window 

Tho caliper win&oww the portion of an image in which tho Caliper Tool eoarehoe for edges. 
It ib defined by the search length and projection length, wnicn are tne widtn and height 
resDectively. of the window in an image that will be projected into a one -dimensional image. 
The caliper window is illustrated In Figure 107. 
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Edge detection 



Figure 107. The Caliper Tool window 

The pixel in the center ot tne caliper window is called the application point This is the point 
in the run-time image at which you want to locate edges. 



Caliper Window Rotation 

For many applications edge information is needed at more than one angle in an image. For 
example, to inspect a rectangular part, you may want to measure the dimensions ot the part 
by oroiecting the image first at 0" to measure length, and then at 90° to measure width. Tho 
caliper window can be oriented at any angle to locate edges in an image. Angles increase 
In the clockwise direction from the horizontal axis. 

Figure 108 shows how changing the caliper window angle affects the protection and edge 
search directions at 90" rotations. The Caliper Toot runs faster for 90* rotations man tt does 
for arbitrary rotations. 
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F/gure 705. The caliper window at 90' rotations 



Skewed Window Projection 



Figure 109 shows a -15° caliper window with window rotation disabled. In this case, the 
Caliper Tool creates a "skewed' winaow end projects it along the angle of skew into a one* 
dimensional image. 
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Figure 100. Skewed window protection 

When window rotation is disabled, you specify how the skewed window will be orojected; 
with or without interpolation. Interpolation increases accuracy at me expense of execution 
speed. 

When you use a skewed window without interpolation for proiectiop, the pixels in the source 
image are summed along tne angle ot skew aa shown in Figure 110. Each pixel in The 
two-dlmenaonal image contributes to one pixel in the one-dimensional image. In 
Figure 110, thoce pixole containing the number 1 contribute to the deeiination pixel 
containing the number 1 . and so on. 
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Figure 110. Skewed window projection without interpolation 

When you use a skewed window with interpolation for projection, the skewed window is 
transformed inio a rectangular window oefore projection. The transformation is performed 
by ctp.tranef orm() t which uses several neighboring pixels from the two-dimensional 
image to calculate the pixel value lor the one-dimensional tmage. See Chapter 2. Basic 
Image Processing, in the tmage Processing manual for a complete description of the 
tunooon otp.trarMformQ. 



50 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 



PCT/US97/18268 



CallperTool 4 



Rotated Window Projection 

Figure 1 1 1 chows a -16* caliper window with window rotation QnabioeJ. In tni« coco tho 
Caliper Toot rotates trie two-dimensional image and projects it into a one-dimensionaJ 
imaga. 




Figure m. Rotated window projection 



Choosing the Projection Type 

The choice of skewed or rotated projection depends on the angle of the edge or edge pair 
of interest in the image and the remaining content of the image. If the rotation angle is close 
to 90° or if a skewed image will contain enough of the edge or edge pair of interest to 
generate strong edges, you can use skewed projection to speed program execution. 
Figure 1 12 shows an image, a caliper window and the one-directional image that results 
from projection. The skewed window contains enough edge information, so skewed 
projection may be used. 
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Image 




Figure 1 12, Skewed projection may be used 

Figure 1 13a snows a caliper window where tne edges or interest ars at IS" ana window 
rotation is disabled. A window targe enough to enclose these edges would also include 
much of the dark area in the image, which in some cases would result in a one-dimensional 
image where the edges are obscured. 

Figure 1 13b snows a caliper window at -15 c in me same image with window rotation 
enabled. Only the edges of interest are included in this window. 
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Figure 1 13. Skewed projection (a) ana rotated projection (b) 

Caliper Tool Optimization 

When setting up the variables for a caliper, you specify the following types of optimization: 

• Space optimization: The Caliper Tod is optimized for minimum memory use at die 
expense of execution speed. 

• Speed optimization: The Caliper Tool will use a number of internal techniques to 
improve execution speed at the expense of increased memory use. 

• Default optimization: The Caliper Tool will weigh space and speed equally, using 
©lightly more cpace than if you specify epac* optimization and clighUy clowor 
execution speed man if you specify speed optimization. 
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The Edge Filter 

After projection, an eage mar is run on the resulting one-dimensional image. The edge filter 
accentuates edges in the image ana produces a fttttrod tmaga. Tha peaks in this image 
indicate strong edges. 

The two parameters of the edge filter are its size and laniancy. Edge filter size is the number 
of pixels to each side of the edge to consider in the evaluaoon. Edge filter leniency is the 
ntmber of pixels at an edge to ignore oetween light and dark sides of the edge (leniency 
is described in turtner detail below). Figure 114 illustrates an edge filter. The edge filter is 
positioned over the leftmost pix«l in me one-dimensoruil image where it will entirely fit in 
that image. Pixel values to the left of the leniency regon are summea and subtracted from 
the Bum of the pixel velueo to tho right of tha leniency region: in tnia example. (0-r5)-(0-0) 

yields 5 for the edge filter image. 
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Figure T M. Tho oago tutor 10 poaitionod in tha one-dimanmonai imago. 

As shown in Figure 115. the edge filter is tnen applied at each successive pixel in the 
one-dimensional image until the edge filter no longer fits entirely in the image. At each pixel 
position, the pixel values to the left of the leniency region are summed and subtracted from 
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the sun of the pixol values to the nght ot the leniency region ana the resurt is wntten to the 
edge fitter image. The resulting image will be 2 « W2e + leniency - 1 smaller than the 
one-dimensional image. 
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Figure 1 15. The edge filter is applied at each successive pixel until the 
edge filter no longer fits entirety in tha image. 

Figure 116 shows a graph of the one-dimensional image and the edge filter image from 
Figure 115. PeaK position is calculate a to sud pixel accuracy. 
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1-D image 
Edge filter image 



Figure 1 16. Edge fitter image graph vs. one-dimensional image graph 

The graph in Figure 116 represents en eage fitter applied to an ideal Image; few 
applications would result in as ideal a graph. All Images contain aome degree of noise, and 
a typtoal edge in an Image cart be several pixels wide. The new two sections provide 
guidelines for optimizing the edge fitter by choosing appropriate values for size and 
leniency. 

Choosing Edge Filter Size 

Edge filter size specifies me number of pixets on eitner side of an edge to consider in an 
edge evaluation. Size should usually De greater than 1 because noise in an image causes 
grey level differences between neighboring pixels. Figure 117 Shows a graph Of a 
one-dimensional image of a light band on a dark background, and a graph of an edge fitter 
image where a size of 1 was uaed. Because there is noise in the image, many more peaks 
exist than are of interest. 
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Edge filter peaks 



Figure 117. An sage titter size ot 1 is typically too small. 

increasing size tenets to smootn me eage niter image oe cause summing several pixels on 
either side of an edge tends to reduce the effects of noise. Figure 118 shews an image 
similar to me one in Figure 117 where a size of four woa used. Tne only two peaks in tne 
image are the peaks of interest. Size is typically between 2 and 5 inclusive. 
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Edge filter peaks 



Figure 1 1& Using four for edge filter sue reduces the effects of noise on 
the edge filter. 

Choosing Edge Filter Leniency 

Edge filter leniency specifies a region of pixels to ignore at an eage in an image. Due to 
slight changes in edge or edge pair position or orientation, a pixel on the edge of a feature 
may not always have the same grey value. By setting a leniency rone between the light and 
dark areas of a feature, the effects of sifght changes in feature location on filter results are 
minimized. 

Figure 1 1 9 shows a graph of a one-dimensional image with two edges that span two pixels 
eacn. and graphs of the output of two edge filters. The first edge filter has a leniency of 0, 
the second edge filter has a leniency of 2. The edge filter with a leniency of 2 produces 
higher peaks. Two is a typical value for leniency. 
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Point Registration Tool 

This chapter describes the Cognex Point Registration tool. It contains seven sections. 

The first two sections, this one and Some Useful Definitions, provide an overview of the 
chapter and define some terms that you wip encounter as you read. 

Point Registration Too) Overview provides information about the capabilities and intended 
use of the Point Registration tool. 

Haw tha Point Registration Toot Works provides a general description of the operation of 
the Point Registration tool. 

Comparing the Point Registration Tool with Search compares the Point Registration tool 
with the Cognex Search tool. 

Using the Point Registration Tool describes some of the techniques that you will use to 
implement on application using tho Poinl Registration tod. 

Point Registration Tool Data Structures and Functions provides a detailed description of the 
data structures and functions that you will use to implement your application. 



60 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 



PCT/US97/18268 



6 Point Registration Tool 



Some Useful Definitions 



point registration 

global minimum 
local minima 



A search technique designed to determine the exact point at 
which two images of the same scene are precisely aliened. 

The lowest value in a function or signal. 

A low value in a function or signal. 




Local minima Global minimum 



•ubplxel accuracy Positions within an image may be specified in terms of whole 
pixai positions, in which caaa tha position refers to tha upper - 
left corner of the pixel, or in terms of fractional pixel positions, in 
which case ihe oositian may lis anywhere within a pixel. 
Positions specified in terms of fractional pixel positions are 
referred 10 as having subpixel accuracy. 
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Point Registration Tool Overview 



The Cognex Point Registration tool performs fast accurate point registration using two 
images that you supply. 

Point Registration 

Point registration is the process of determining the precise offset between two imaoes of 
the same scene. 

You use the Point Registration tool to perform point registration by providing a model image 
and a target image, along with a starting point within the target image. The model image 
snouid be a reference image of the feature you are attempting to align. The target image 
should be larger than the model image, and it should contain an Instance of the model 
Image wimin it 

The Point Registration tool will determine the precise location of the model imege within the 
target image. The Point Registration tool wW return the location of this match, called the 
registration Dtint. with subpixa) accuracy. 

Figure 160 illustrates an example of how the Point Registration tool performs point 
registration. 





Model image 



Target image 



Location of model image 
within target image 



Figure 160. Point rggistration 
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The Pomt Registration tool is optimized to locate precisely the model image within the taroet 
image, even tf the target image contains image defects such as reflections or if the mode! 

«^ G J! P ? tiattV > 0b9CU,ad by ° ther featUres in me mr ° eI im « e - 161 iilustratesan 
example of point registration where the target mage is partially obscured. 




Model image Target rmaoe Location of model image 

* within target image 

Figure 161. Point registration with partially obscured target image 
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How the Point Registration Tool Works 

The Point Registration tool finds the location of the model .mage within the target imaae 
You operate the Point Registration toot by supplying the model end target imagee aiona 
WRh me location within the target image where you expect the origin of the model image to 
21 ^f? Re 9; sEral,on <** wll determine, with subpixel accuracy, where the oriojn of 
the model image lies within the target Image. 

The Point Registration tool works by computing a score indicating the degree of similarly 

«^ Mn ^ e r ,m T !? e and 3 Dartjcular P°«ion of the target image that is the same size 
aa the model image. This score can range from o to 253, with a score of 0 indicating that 
l ^^ 8fl and ^target mage are perfectly similar and a score of 255 indicating 
that the model .mage and the target image are perfecUy dissimilar. The tool computes this 
score tor locations in the immediate neighborhood surrounding the starting point. The tool 
will find the location within thio na.gnborhood of me target image that produces me focal 
minima in the value of this score. wwwiBigai 

By adding an interpolation step, the Point Registration tool then determines the location of 

2! T= imflSe W,,h,n * e rar90t ima9e wiln 3lJ bPi^l eccurecy. For typical images the 
Point Registration tool can achieve accurate registratron to within 0.25 pixels. 

Because of the way the Point Registration tool aeete the local mm™, if the starting point 

you speciiy is more than a lew pixels from the actual registration point, the iool may not 
return the correct registration point. The exact amount of varienoe that the Point 
Registration tool can tolerate will vary depending on the images. The variance may be as 
small as 3 to 5 pixels for some images or as large as 30 pixels for others. 

Point Registration Score 

Each time it » invoked, the Point Registration tool will return. In addition to tne location of 
the origin of the model image within the target image, a score indicating the degree of 
«m.. a r.ty «™ modo. mag* and the target image. The ecore returned by the tool 

will be from 0 to 255, wiih a score of 0 indicating that the model image and the target image 
are perfectly similar and a score of 255 indicating that the model image and the tarqet 
image are perfectly dissimilar. 

If your point registration receives a high score, the actual precision of the point registration 
location may be somewhat lower than if the point registration receives a low score. 

Exhaustive Point Registration 

When you use the Point Registration tool, you supply the tool with two images a nd a starring 
location within the model image. The tool will confine .ts point registration search to a small 
area around the starting location that you specify. 
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The Point Registration tool can also perform point registration exhaustively, that is. by 
computing the score for every possible location of the model image within the target image. 
This procedure, called exhaustive point registration, is extremely slow. It can be helpful, 
however, in debugging applications where the Point Registration tool does not appear to 
be working correctly. 



Masked Point Registration 



You can limit the areas of the model image and target image that the Point Registration tool 
use* to perform the point registration by supplying a series of rectangles to the tool, if you 
supply these rectangles, the tool will compute the score based only on those pixels 
contained within the rectangles that you specify. 

Figure 162 illustrates an example of specifying a masking rectangle. In this example, the 
right side of the model is often obscured in the target image. By specifying a rectangle that 
oovera the left eide of the modal Imege, you can cause the Point Registration tool to 
consider only the pixels in that part of the model image. This will tend to increase the 
accuracy of the point registration. 



Masking 
rectangle 





kJ im „„„ -j- . - Location of model Image 

Model image Target image ^ thin fArnnt ■ ft y 



within target image 
Figure 162. Masked point registration 

You can ai90 specify several masking rectangles. Figure 163 illustrates an example where 
the oentor of the modal imago a often obscured in the target image. By sp«=ifyjng a series 
of rectangles, you can eliminate from consideration the part of the Image that is most 
frequently obscured. 
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Masking 
rectangles^^J 




Model image 



Target Image 



Location of model image 
within taroet image 



Figure 163. Ma ckoct point rogiatration u&ng multiple rectangle* 

If you specify multiple masking rectangles and the rectangles overlap, any pixels that are 
contained in more than one masking rectangle will be counied toward the score multiple 
times, once for each rectangle in which they are contained. 

In ail cases, the Point Registration tool will return the location of the origin of the model 
(mage within ihe target image. 



The Point Registration tool may not work well in cases where the model image and the 
target image have different intensity values. You can perform limited image processing as 
part of the point registration by supplying a pixel mapping table. 

It you supply a pixel mapping table, every pixel in the target image will be mapped to the 
value in the pixel mapping table at the location within the pixel mapping table given by the 
pixel's value. For example, if a olxel in the target image had a value of 300, ft would be 
mapped to the value of the 20O" element within tne pixel mapping table. 

If you nave specified a masking rectangle, only those portions of the target image that lie 
within the masking rectangle or rectangles will be mapped through the pixel mapping table. 

For more information on pixel mapping, see Chapter 3, Pixel Mapping, in the Image 
Processing manual. 



Image Normalization 
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Comparing the Point Registration Tool with Search 

This section compares the Point Registration tool with the Search tool. For more detailed 
information on the Search tool see Chapter 1. Searching. 

The Point Registration tool has the lollowing advantages over the Search tool: 

• l! is less sensitive to images that contain areas of pixels that have widely different 
values than the same pixels in the model image (i.e.. blotches or occlusions). 

♦ It requires no training step. 

The Point Registration tool has the following disadvantages when compared with the 
Search tool: 

* It is capable ot line alignment only. 

• It does not work with scaled or rotated target images. 

♦ tt does not work well with images with brightness changes. 

• It will not find or rank multiple registration Ddnts. 
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Using the Point Registration Tool 

This section describes how to use the Point Registration tool. 

Obtaining Model and Target images 

You acquire the images thai you use as model and target images using the techniques 
described in Chapter 1.3000/4000 Acquisition and Display, in the 3000/4000 image 
Acquisition and I/O manual and Chapter 1. 5000 Acquisition and Display, In the 5000 
Image Acquisition ana I/O manual. 

You should acquire all images that you plan to use with the Point Registration tool, 
especially images of scenes with low contrast, using a bit depth of at least 8. For fastest 
performance, you should acquire images as FASTS images. 

When you acquire the model image, you should take care to ensure that the image is as 
ideal a model as possible of the images that you will be using as target images. The Point 
Registration tool is designed to align target images with a wtde variety of image defects, 
but in order for tho tool to work, tha model imago naada to be ae free from defects aa 
possible. 



Specifying a Starting Offset 

When using the Point Registration tool, you must specify a starting location In the model 
image. The tool will perform its point registration search starting at the point that you 
specify. Because the tool only seeks the local minima for the score value, if you specify a 
ctarting location that vaneo grootfy from the actual registration point, the point registration 
operation will fail. 

You can use other vision tools such as the Search tool or the inspection tool to locate the 
model feature within the target image, or you can rely on operator input to specify the 
coarse location of the feature. 
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Point Registration Tool Data Structures and 
Functions 



This section contains descriptions of the Point Registration tool data structures and 
functions. All Point Registration tool data structure and function names begin wim me prefix 

Structure creg_params 

creg_pwwna contains parameters that determine how the point registration search will be 
conducted. A structure of this type is suppllea as an argument to ereg_jx>lnt_r»grsterO. 
•include ^register . h> 

cypadef struct 
{ 

c_lnt32 Btart_x, 
otajrt_y; 

c_ Inc.32 roct_count; 

char 'optional _^iapf 

c_Int32 £lag=; 
) crea^paratne ; 

• starts and stan_y are the x- and y-coordinates of the starting position for the point 
registration search. 

rect^count\s the number of rectangles in the rects argument to cr*Q_polnt_reglatftrO- 

• opVonatmap is an optional pixel map. If optionaLmap is non-NULL, the Point 
Registration tool will replace each pixel in the target image with the value contained in 
the element of optionai^map corresponding to the value of the pixel in the target 
imago. If you supply a value for optionaLmap. you muet eupply an imaqe of type 
FAST8. 

You can use optionai_map to compensate lor image-to-image brightness variations. 

flags specifies the type ol point registration mat me Point Registration toot will perform. 
flags is constructed by ORing together any of the following values: 

• CREGJTYPE 1 is reserved for future use ; you should always include CREG_ TYPE 1 
in me value of flags. ~ 

• CREGJTYPE2\z reserved for future use; you should never include CREGJTYPE2 
In the value of flags. 

• CREG^NORMAL is used to specify a point registration based on finding me local 
minima of the acore value. 
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• CREG.EXHAUSTIVE is used to specify a point registration based on finding me 
global minimum of the score value. 

If you specify CRE G_NORMAL and if sterur and starx_y are more than a few pixels 
from the actual registration point, the tool may return a location that does not represent 
tne Desc maich within me Image. If you specify CREQ_EXHAUSTIVE t the registration 
will return the best registration match for the entire image. An exhaustive point 
registration will be extremely stow. 

Structure creg results 

ereg_POlm n r«gl«ter returns a pointer to a creg_reeutU structure. cr»gj>o|m_regUUr 

fills in the structure with information about the results of a point registration operation. 

» Include <register.h> 

typedef struct 
( 

float x, 

y; 

c_Int32 score, 

time ; 
char on_edge_x. 
on_edae_y ; 
creg_rasultG; 

x and y are the x-coordinate and y-coordinate. respectively, of the location within the 
model image at wnicn the target image was found, x and y give the position with 
subpixal accuracy. 

score is a measure of the similarity between the pixel values in the model image and 
the target image at tne nearest whole pixel alignment position, score will be between 
0 and 255, with lower values indicating a greater degree of similarity between the 
model image and me target image. 

time is the total amount of time that the tool spent on this point registration, in 
milliseconds. 

ort,adQo_x is sal to a nonzero value if. along the x-oxic. on© edge of the model image 
area is at the edge of the target image. If on_edge_j( is nonzero, the accuracy of the 
position information may be slightly reduced from whet it would otherwise bo. 

or^edge^y is set to a noruero value if. along the y-axis, one edge of the model image 
ere© 19 at the edge of the target image. If on^edge_y Is nonzero, the accuracy of the 
position information may be slightly reduced from what it would otherwise be. 



} 
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Function creg_point_r©gistor() 

creo_peUnt_reai«tw() performs pomt registration ufting the model imago and target image 
mat you supply. The point registration will be controlled by the parameters in the 
crea-J>ftrama structure supplied to the funcoon. crog_polnt-r»gl«ter() returns a pointer to 
a cr»g_renjtt» structure that describes the result of the point registraDon. 

t include <register.h> 

creg^resulce •crag_poinc_regietex (consc eip_bu«fer 'target, 
const cipjauffer "model, const cia_rect *rects, 
consc creg_param6 "paramo, creg_resulcs 'results) ; 

• target points to the image to use as the target imago for this point registration, target 
must be at least one pixel larger in both the x-dimension and the y-dimension than 
model. 

• model points to the image to use as the model image for this point registration, model 
must be at least one pixel smaller in both the x-dimension and the y-dimension than 
target 

• reels points to an array of cla.rect structures. 11 rects is non-NULL. the Point 
Registration tool will limit the comparison of pixel values to just those pixels that lie 
wtmin tne rectangles contained in rects. If the rectangles in rects overlap, a pixel will 
be included in the comparison once for each rectangle that contains it. 

• params points to a cregjMrams structure that defines the parameters to use for this 
point registration. 

• results point3 to a creg_j Mutt structure, cr»g_polnt_reg|*terO will place the results 
of this point registration m rasults and will return a pointer to roeulte. 

If you supply NULL tor results. crog_polnt_r«gf«terO will allocate a erog_reeulf 
structure from the neap using the default allocator. You can free this strucrure by 
calling free(). 

cnjg_poirrt_regl«tor() throws CG£N_ERFLBADARG it 

• reefs is NULL and the rect^count field of params is not 0 

• If model is larger than or the same size oa target 

• If the start jc field In params is less than 0 or greater than the width of target minus the 
width of model 

• If the stanjr field In oaramsts less than 0 or greater than the height of target minus the 
height of model 

• If model is NULL 

• If params is NULL 
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♦ If raz^ef is NULL 
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3 Line Finder 

An Overview of the Line Finder 

The Line Finder computes the locations of Iine3 in an image. For each line that it finds, it 
renins an angle e and a signed distance dfrom the central pixel of the image. Given a and 
d t you can compute all points (x,y) in the fmage that satisfy the equation of the line (as 
illustrated in the section The Transformation from Cartesian Space to Hough Space on 
page 204). The algorithm thai the Line Finder uses, along with the angle/distance 
classification of lines, is explained in How the Line Finder Works on page 166. 

You can use the Una Finder in either of two modes, quick or normal. In quick mode the Line 
Finder provides tne location or me point along eacn tine mat is nearest to me center or me 
image, along with the angle of the line. In normal mode, the Line finder also estimates the 
length and density of each lino mat it locates, using statistical methods, and returns the 
position of the center of mass of the line. 

Figure 78 illustrates the two modes of the Line Finder. The Image In Figure 78a Is shown in 
Figure 78b after the Line Finder nas been invoked in quick mode. The presence of the four 
lines is indicated. Each tine is drawn to the screen at a constant length, but no attempt is 
made to estimate its actual length, figure 78c shows the result when using the Line Finder 
in normal mode: the length of each line segment is estimated. 

For a detailed explanation or me data returned by tne urte Finder in eacn mode, see tne 
section Results in Quick Mode and Normal Mode. 



b c 

Figure TO. An image (a), tines round in quick mooe (Q), ana tine 
segments found in normal mode (c) 

The Line Finder finds lines regardless of grey scats variations In the Image, rotation of the 
scene, or changes in scale. A typical application tor the Line Rnder is locatino fiducial 
marks in a series of images In which grey levels, scale, and rotation are unpredictable from 
one image to the next. Figure 79a and Figure 79b illustrate two images, each containing 
the same fiducial mark: a square within a square. However, the sizes of the marks, and their 
rotations differ Between ine two images. Also, the mark in Figure 79a is a dark square within 
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a lighter square against an even lighter background, but the mar* in Figure 79b is a light 
square within a darker square against a darker background. As shown in Figure 79c and 
Figure 79d. the Line Finder finds the outlines ot the liduciat marks despite these variations. 




d 



Figure 79. The Une Finder is immune to grey sea/e, size and rotation 
changes 

Since tne une Finder is an intemieaiaie-ievel toot, your application will prooaDiy require 
that you do further processing, using the results of tne Line Finder as input. You will 
probably not be searching for linos in your image, out rather for a feature wnose presence 
is implied by the locations of lines. For example, in Figure 79. me Line Finder does not 
return the locations and dimensions ot the fiducial marka. only the locations of the lines that 
bound the nested squares. Computing the precise locations and sizes of the fiducial marks 
requires post-processing based on the Line Finder's results. 

The Line Finder demo code includes post-processing of this sort. It contains functions that 
locate squares of any si2e and rotation, using the Line Finder. This code deduces the 
presence of squares based on the Line Finder's results, eliminating extraneous lines. See 
Chapter 1. CognexAPt Introduction, in the Development Environment f manual for the 
location and name of the Contour Finding demo code on your system. 
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How the Line Finder Works 



This chapter explains hew the Line Finder finds lines in an image. This discussion is divided 
into the following cubcectione: 

• Defining a One by Angle and Distance describes the definition of a line used by the 
Line Finder. 

• Hough Space introduces the two-dimensional space in which the Line Finder records 
lines. 

• The Hough Lino Transform describes the algorithm that the Line Finder uses to find 

lines In an Image. 

The Line Finder uses a Cartesian coordinate system whose origin is the center of the image. 
Most of the examples in the book are drawn in this coordinate system, atthouoh a few use 
image coordinates. These two systems are pictured in Figure SO. Notice that the x-axes of 
the two systems are the same, but the y-axes are opposite. Note also (hat positive angles 
in Cartesian coordinates are measured counterclockwise from the x-axis; positive angles 
In image buffer coordinates are measured clockwise from the x-axls. 



-y 



-x 



Image buffer 




coordinates 









tX 



Caaesjan 
coordinates 









+x 



+y -y 

Figure 80. Image buffer coordinates and Cartesian coordinates 

The Line Finder uses the central pixel of the image as a reference point. The image buffer 
coordinates of this pixel fa. yj are computed with integer division, using the following 
formulas: 
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y c - H/2 

where ivis me widin or me image and Wis tne Height ot the image, in pixels. In the Line 
Finder, the Cartesian coordinates ot the central point in the image are (0,0). 

Defining a Line by Angle and Distance 

The Line Finder defines a line On Cartesian coordinates) by its angle from the x-axis and by 
the signed distance from the center of the imago (the central pixel) to the line. These two 
parameters uniquely describe any line in Cartesian space. 

The feature in Figure 61a contains an edge that lies along a line (Figure Bib). The line is 
defined by its angle from the horizontal (6) and by the shortest distance from the center of 
the Image to the line (Figure 81c). Note that the distance vector is perpendicular to tne line. 




a b c 



Figure 8 i. Angle ana distance define a line 

In the definition of a line by angle and distance, the distance is negative if the angle of the 
distance vector (in Cartesian coordinates) is greater than or equal to 0° and less than 1 80°: 
the distance is positive if the distance vector has an angle greater than or equal to 1 80° and 
less than 360*. 

Distance is signed to avoid the ambiguity illustrated in Figure 82. Figure 82a shows an 
image witn wo features. Each feature is bordered on one side by a line. Aitnougn tne lines 
are different, they have the seme angle 6 and the same absolute distance c/from the center 
(Figur© 82b). 
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c d 



Figure 82. Tna angle of the distance vector determines the sign of tha 
distance 

By designating a sign to the distance vector, based on its angle (5 In Figure 82c, 5+ 180 in 
Figure 82d), each line is ensured a unique definition. In this example, the two lines are 
defined as (8. d) and (9. -d). 

Once you Know the distance and the angle ot a line, you can compute all points (x» yj. in 
Cartesian coordinates, that tie on the line. Given a distance, d and an angle 6. a line is all 
points (Xo. y e ) that satisfy the following equation: 



d =s XgBinO- y 0 cc*6 

Th* derivation of thie formula ia supplied in the aootion Tho Transformation from Cartesian 
Space to Hough Space on page 204. A3 an example of its application, if 9 is 45 and d is 0, 
a line is all points in Cartesian space that satisfy the following equalities: 
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0 «= xain(45)-ycos<45) 
1 1 

x = y 

Hough Space 

Hough space is a two-dimensional space in which the Line Finder records the lines thai it 
finds in an image. A point in Hough space represents a line; one axis represents angle, the 
other represents distance. 

Hough space is Implemented in the Line Finder as an array: a simple example is shown in 
Figure 83. The Hough space in this example can record eight angles and nine distance* 
and therefore 72 lines. 

The Line Finder lets you control the size of Hough apace. You specify the range of distance 
in pixels, and, in the edge detection data structure that you pass to the Line Finder, you 
supply the number ot angles that the Line Finder can compute. 
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Figure 63. Hough space 

The Hough space in Figure 84 contains a single line. It has an angle of 135° and is a 
distance of 3 units from the center of the image. 



79 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 



PCT/DS97/18268 



3 Line Finder 



Angle 



315 
270 
225 
160 
135 
90 
45 




-4 -3-2-1012 
Distance 



Imago 



Figure 84. Hough space containing a single line 



The Hough Line Transform 



The Line Finder uses the Hough Line Transform to locate lines in an image. This method is 
outlined and illustrated in this section. To understand this discussion, you should be familiar 
with edge detection and peak detection, as described in Chapter a, Edge Detection Tool, 
in the Image Processing manual. 

The Hough Line Transform, as implemented by tnis tool, takes advantage of the following 
rule: (f an edge pixoi's angle 9 and location (xO.yO) is Known, tnen the lino on wtiich the pixel 



lies is also known: it is the line with angle *+90 that contains the pixel (xO.yQ). This is 
illuatratod in Figure 66. 



170 
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b c 



Figure 85. Four eages in an image (a), eage angles at Tour eage pixels 
(b). and tine angles at those pixels (c) 

The Hough Line Transform is implemented as follows: 

1 . Using edge detection, the Line Finder creetes en edge megnitude image end an edge 
angle image from the input image. 

2. The Line Finder creates and dears a Hough space. 

3. For each edge pixel m the input image, the Line Finder does the following: 

• Using the edge angle t and location (xO,yO) of the pixel, the Une Finder calculates 
the line's distance from the center, d. and its angle. 6 (using the formula 8 ■» o * 
90). 

• The Line Finder increments the bin in Hough space representing the line (e.d). 

4. Once all edge pixels in the image have been examined, the Line Fnder searches 
Hougn space for maximum values, using peak detection. The highest value in Hough 
apace represents the strongest line in the image, the second highest represents the 
second strongest fine, ana so forth. 

5. The location of each line ia primed out and, optionally, each line is drawn in an output 
image. 

Figure 86 through Figure 89 illustrate this method. 

Figure 66a is an image passed to the Line Finder. Figure 66b is a stylized edge 
magnitude image created from the input image. 
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b 



Figure 60. An input image (a) and the edge detected input image (b) 

Each edge pixel is processed, as described above. For example, because the outlined 
edge pixel in Figure 87 belongs to ihe line (90. -2), trie bin in Hough space mat represents 
that line is incremented. 
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Figure 87. The bin in Hough space tor the edge pixel's line is 
incremented. 
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Line Finder 



Figure 88 shows the final Hough space along with the Input Image. Figure B9 shows the 
peak-detected Hough space, with each of the peaks outlined. Figure 89 also snows the 
input Image with the lines, represented by the peaks In Houah space, drawn into the input . 
image. 
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Figure 88. The final Hough spaca and the input imaga 
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Figure 09. The peaK-detectcd Hough space and tho input Imago, with 
lines added 

Notice that the Hough space contains many extraneous lines. If. lor instance, tne vision 
problem is to find the triangle in the image, the tower scoring lines that border the blob need 
to be eliminated. Almost all of the Une Finder applications require post-processing of this 
sort. In this example, simply locating the three highest peaks in Hough space eliminates 
the unwanted lines (Figure 90). In most applications, the required post-processing is more 
complex. 
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Figure 90. The output image after post -processing 
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APPENDIX H 



Patent Application for 



MACHINE VISION CALIBRATION TARGETS AND METHODS 
OF DETERMINING THEIR LOCATION AND ORIENTATION 



Software Listings 



THE FOLLOWING APPENDIX IS NOT BELIEVED TO BE NECESSARY FOR 
ENABLEMENT OR BEST MODE DISCLOSURE OF THE INVENTION DISCLOSED AND 
CLAIMED IN THE ACCOMPANYING APPLICATION. 



86 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 



POYUS97/18268 



* Copyrighc (c) 1996 by Cognex Corporation, Natick, MA USA 

* All rights reserved. This material contains unpublished, 

* copyrighted work, which includes confidential and proprietary 

* information of Cognex. 
* 

* cwa_find_ target . c , Aaron Wallack - 13 Apr 1996: Created. 

* Time-stamp: <96/07/30 09:32:47 awallack>: last modified . 
* 

* SRevision: /main/noneS 
*/ 

# include <prealign.h> 
^include <math.h> 
f* include <stdio.h> 
# include <cct.h> 
^include <cip.h> 
1* include <ctm.h> 
^include <caq/caq_def . h> 
H include <cwa_priv.h> 
^include <search/ctr_de£ . h> 
^include <search/cse_def . h> 
# include <clp/cclp.h> 
^include <clp/ cclp_er . h> 
# include < cr.p . h> 

/ * This file includes a function fcr local inner a crosshair :vc 

* square) target. 

+ — -ii ** — — — — 

* 8 43 Sri 3 3* 

# ******* 
#3###sss*s**i*#i*-3 



* The algorithm is intended to handle the cases when the target is 

* imperfect (occluded or broken) 

* (occlusion) 



87 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 



PCT/US97/18268 



broken ############# 
############# 

•fhese routines are intended to be used as a subroutine for 
determining the cameras' field of views (with respect t2 an absolute 
coordinate system defined by a motion stage) . 

The target localization routine {cwa_f ind_target) takes as input: 

img - a cip_buffer containing an image of the target 

calib - a calibration structure intended to characterise the 

non- rectangular ity of the pixels 
angle - the orientation of the crosshair ir. :ems of image 

coordinates (we require this 

orientation escimace t c be accurate z z 2 degrees . - . ? 5 radians 
result - pointer to a cwa_lir.e structure where "he resul z will be 
returned 

Requirements : 

Image quality: 

The image of the crosshair muse be visible and perfect 
(without occlusion or breaks'; throughout a 100 X IOC pixel square 
surrounding the crosshair center. 

Consequently, the center of the crosshair t.us: be az least 
50 pixels away from the borders c f the cip_cuffer. 

Orientation estimate accuracy: 

The given angle estimate (given in terms cf image cccrdina tes ) must be 
accurate to within 3 degrees 0.05 radians 

This algoritm tolerates occlusion and breaks in the -arget 
so long as these imperfections occur at least 50 
pixels away from the crosshair center 

Major Consideration: 

One interesting thing to note about crosshair calibration, and the reason 
that we use fcur distinct rays, is that the separation between the opposite 
rays depends upon the lighting and aperture of the lens. Furthermore, we 
cannot assume that the separation between these edges will remain constant 
from one setup to another 

what we really want is the position of the ' * ' 
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############### 

############### 
############### 
############### 



Since we do not know what the of f set/ separation is. it turns out that the 
repeatability of the entire fit (localization of the crosshair} is bounded 
by the accuracy of the smallest of these four lines. It doesn't make any 
sense to include 10000 edge points from one ray if we only have 50 edge 
points from another (the localization error will still be on the order of 
0 . 1 / sqrt ( 50 ) pels, assuming random i.i.d. errcr and s-rpixel localization 
accuracy of 0.1 pixel). 

We want to incorporate the same nunier of edge points frcr. specs it e rays 
because we always want to fit the lines cc -he same exact points for 
highest accuracy and repeatability- . When the calibration target is 
occluded by a particular obi ec: [such as the borders that can be seen in 
the database;- , then if we use the exact sane number of pixels cn the 
opposite unocciuded ray as the number of pixels on the occluded ray. then 
we will always be locking at the same pixels cn the unocciuded ray 

Basically, since we do not know which edge points correspond to 
imperfections ir. the target and which edge pcints correspond to 
valid points on che target, we use a boot - scrapping algorithm to 
compute regions where edge points are 'J?JL1Z . 

Assuming that the target is perfect within the GUARANTEED GOOD 
REGION (center +' - 50 pixels) . we can compute four GCCD SAYS 
characterizing the four rays emmating from the crosshair center. 
Then, we can use these GOOD RAYS to define the VALID REGIONS (the 
set of points within a TIGHT_THRESKOLD of the GCCD RAYS) . Finally, 
we improve the estimates of the four rays by going back and 
enumerating edge points along the GCCD RAYS 'until we encounter an 
edge point which falls outside the VALI- REGIONS. 

Definitions 

GUARANTEED GOOD REGION 

region surrounding the origin where we insist that the calibration target 
must be perfect 

GOOD RAY 

A ray fit to points enumerated within the guaranteed good region 
VALID REGION 
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Region surounding good ray where we cr-sc chac -he edge points are due to 
the crosshair and not from occlusion or breaks 

OUTLIER POINTS 

Points which we cannot trust because they do not fall viih;- the valid 
region 

CENTER OF CROSSHAIR TARGET 

V outlier 

V edge points 

< good ray > 

######## 

VALID REGION 

: < guaranteed gccd region > 

We are adopting a better-safe - thar. - scrry stracsgy for handling 
occlusion (as soon as we see a "bad" pcmt. we lose faith ir. all of 
the subsequent points) . Although this approach is suicpti-al . the 
database images seem to contain nearly perfect points emmattng 
from the origin that it should work well ir. practice. 

Algorithm: 

1} Use absolute normalized correlation search to approximate! y 
locate the crosshair center 

This involves constructing a synthetic model of a rotated 
crosshair center (see grcss_lcca te_crcsshair_cer.ter_in_i:r»age ■ '. 

2) Given this initial position estimate and the user -supplied orientation 
estimate, improve the positional and orientacional estimates by localizing 
four good rays eminating from the crosshair center. This involves 
enumerating edge points within the GUARANTEED GOOD REGION (50 pixels) 
along the supplied orientation as well as 

theta+PI/2, theta+PI , theta+3*PI/2 . These computed four GCCD RAYS are used 
r in step 3 to check whether edge points are outliers. In other words, these 
' four good rays define VALID REGIONS separating good edge points (points we 
r believe correspond to the crosshair) from bad edge points (points we 
' believe are due to occlusion or breaks) We should also point out that we 
r fix the orientations of the four rays by "averaging" these orientations, 
J i.e., we incorporate the constraint that the four rays correspond to 

* orthogonal physical lines. (see 

* compute center_position_and_angle_assuming_f our_guarantsed_good_rays ( ) ) 

* 2) Go back and enmerate edge points along the four rays until we find an 

* edge point which does not satisfy the VALID REGION constraints. We also 

* throw out some of the edge points in order to realize equal numbers of 

* edge points from opposite rays. (see 
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compute_center_positicn_and_angle_aiong_guaranteed_gccd_rays ; I : 
NOTES 



When we enumerace edge points along rays eminacing from -he origin . 
we starr out a small distance away from the origin sc that we don't 
get confused by two different edges (this buffer zone is 
characterized by the constant C;RCUlJ\J^_BUFFER_-ZONE_ARCUND_CENTE?. , 
which is 10 pixels) 

* The central function in this algorithm is 

* enumerate^ points_and_f it_line__along_ray { ) which enumerates edge points 

* along a specified path. enumerate_points_and_f it_line_along__ray ( ) applies 

* calipers at sample points along the path (the edge estimates car. be 

* further improved using parabolic subpixel interpolation; Along a specific 

* path. enumerate_points_and_f it_iine_along_ray { ) always makes integral unit 

* steps in either the x or y direction. This is sc that we can apply 

* axis -aligned calipers along ever*/ row or column when -he seep s::e is I 

* enumerace j3oir."s_ar.d_:i:_i:r.e_alcr.g_ray ; : er.ume races edge points 

* until it either encounters a bad edge pemt, cr finishes sampling 

* the edge segment. At this peine, it ceases enumerating edge points. 

* and fits a line to the points. There are f=ur ways an edge pemt 

* can be "bad". If enumerate_pcmts_and_f it_line_alcr.g_ray (; 

* encounters any of these four situations, it returns 0 after fitting 

* the line; if it successfully enumerated all of the edge points 

* along its specified course, it returns l. 

1. no edge is found by the caliper applied at the sample point 

2. more than one edge is found by the caliper and the highest 
scoring edge is less than twice the score of the second highest 
scoring edge 'this 2 X comes f r err. -he CONF T JSZCN_THRESHCLU; 

3. the computed edge point is an outlier (the distance between the 
edge point and the nominal line is larger than threshold) 

4. the caliper extends outside of the cip_buffer 

* When we call enumerate_poir.ts_and_f it_line_aiong_ray { ) in step 1. we 

* specify a ray from a point near the crosshair center tc a point SO 

* pixels away from the crosshair center. We do look at the return vai 

* to check that the enumerate_points_and_f it_line_along_ray t ) did not 

* encounter any problems . 

* When we call enumerace_points_and_f i t_line_along_ray '■ ) in step 2 , we 

* specify a ray from a point close to the crosshair center along a 

* good ray to a point 1000 pixels away (guaranteed to be outside the 

* cip_buffer) . Thereby, even if we do not see any "bad" points, we 

* will undoubtedly leave the cip_buffer at some point. 

* The reason we pass a calibration object to cwa_f ind targec is that we 

* recompute the orientations of the four rays by incorporating the knowledge 
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chat chey all ccme from ort.hogon.al lines. If the pixels are ncn- scruare 
(i.e., by virtue of the fact that the CCD elements are r.cn - square : . then 
we need to account for this when we "average" the fcur orientations. We 
"average" the four orientations by transforming the image orientations 
into physical coordinates (and then truly averaging the physical angles; 
and then transforming the average angle back into image coordinates. We 
try to use the term "phys" to denote physical coordinates and -img" to 
denote image coordinates (such as phys_orient and img_crient ) . 

We use different sized calipers for steps 2 * 3 because we make 
different assumptions about the relative accuracies of our caliper 
application points. We use the terms LOOSE and TIGHT to refer to the 
calipers; LOOSE for step 2 where we have little faith in the 
application points, and are thereby forced to use broad calipers, and 
TIGHT for step 3, where we have much more faith in the application 
points, and can thereby use relatively tight calipers. 

We expect the initial search position to be accurate zz 2 pels, and 
the orientation estimate to be accurate to 0.35 radians. At a caliper 
application point 50 pels away, the caliper rr.ay be ziz by up zz 5 
pels from the correct edge position. Just to be safe, we use a 
caliper with project: ion length 2 4 ; 12 cr. each side; zz enumerate -2 dee 
points in step 2 

After improving the position and orientation estimates ir. siep 2. we 
basically assume that we've got the right positicn to - - 0.2 pels 
and + / - 0.01 radians. Thereby, we can use these lines zz threshold 
outliers because we would expect errors of only 2.2 pels at 200 
pixels away. This is pretty tight because we use a threshold of 2.0 
pels to detemme outliers 

(actually, the 0.2 pels. C.01 radians could significantly impact the 
performance because the f 1 rs z SO pels and the or lentaticr. estimate 
end up predominantly affecting which edge points are included in the 
final line estimates: 

Main subroutines - 

gross_locate_crcsshair_center_in_image : ; 

construct a synthetic model of the crosshair and use search to 
estimate the crosshair position 

f it_line_to__points ( } 

fit a line (x,y,t! to an array of points 
compute_subpixel_position ( ) 

use parabolic interpolation at the neighborhood of the application point 
to compute the subpixel edge position 

enumerate_points_and_f it_line_along_ray ( ) enumerate edge points along a 
line segment from start_point to end_poinc and possibly store the found 
edge points in an array. Furthermore , a nominal line and a threshold are 
passed in as arguments, and when the found edge points deviate from this 
nominal line by mere than the threshold, enumeration is stopped and a line 
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* is fit to the enumerated edge positions. Enumeration is also stopped when 

* two edges are found (confusion) , no edges are found (occlusion; or when 

* the caliper is applied out of bounds (out of bounds.' 

* average_four_image_orientations i) 

* average the orientations of the four rays to compute the best 

* composite orientation estimate. This function transforms the image 

* orientations into the physical domain in order to exploit the 

* constraint that the physical lines are orthogonal 

* compute^encer^ position_and_angle_assuming_four_guaranteed_good rays ( ) 

* improve estimate of the crosshair position and orientation by 

* searching four regions of guaranteed good length (50 pixels) 

* eminating from the crosshair center. Lines are fit to the edge 

* points along the four regions, and then the orientations of these 

* lines are "averaged" . Two orthogonal lines are computed by 

* averaging pairs of the four rays Finally, the position estimate is 

* computed by intersecting the two orthogonal lines 

* compute_center_positicn_ and_ar.gle_alcng_gT-iaranteec_gccd_rays ; i generate 

* final estimate of crosshair position and orientation by enumerating points 

* along four rays (using four gccd rays as thresholding lir.es . We use 

* tighter calipers (than we used :r. step 2*. because we -rust the petitions 

* of the good ravs . 
V 

ttdefine N_EDGES 10 

#define guaranteed_gcod_s i ze so 

^define GROSS_TARGET_SIZE GUARANTEED^GCOD^SIZE 
#define NUM_S AM PLE_ POINTS 1000 
ftdefine LOOSE_TK3ESHCLD 12. 
Jtdef ine TIGHT_TKRESHCK~; 3 . 
itdef ine LOOSE_CXL:pSR_WI^TH 24 

/* ( (int) 2* iLCCSE_THRESKCLD+GUARAir:EED_GCCC_SI2S*sin (0. 25, ] ) - 2 (kernel) * , 

itdefine TIGHT_CALIPER_WIDTH 14 

/* ( (int) 3*TIGHT_THRESHOLD) * 2 (kernel)*,' 

/* 

* TIGHT_CAJLI?ER_WIDTH should be larger than the minimum width so that 

* we check whether there is a sharp edge just outside the 

* TIGHT_THRESHOLD ranga, because if there is a sharp edge just 

* outside the TIGHT_TKRESHOLD range, then we want to declare 

* confusion and stop enumerating ooints 
*/ 

^define LCCSE_CALIPER_PROJECTION_^ENGTH 3 
^define T I GHT_CAI. X P ER_PRO JE CTI ON_L ENGTH 5 
ftdefine MAX_NUM_ POINTS 10 00 
ftdefine EPSILON_THRESHOLD 0.05 

^define CIRCULAR_aUFFER_2ONE_AR0UND_CENTER (8 . +-LOOSE_CALIPER_?RGJECTICN_LENGTH) 
ttdefine STEP_SIZE_USED_TO_ENUMERATE_POIHTS_IN_GOOD_REGICN 3 
^define STEP_S 1 3E_USED_TC_ENUMERATE_ALL_ POINTS 1 
#define MAX_LENGTH_DIAGONAL_ACROSS_CIP_auTFER 1000. 
#def ine CONFUSION THRESHOLD 3 . 
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/* number of pixels to ignore when we run intc an outlier * 

^define OUTLIER_BUFFER 7 

extern int cd_showas ( inc , char*) ; 

extern double cwa_poinc_distance ( ) ; 

char* cy_prealign_cwa_f ind_carget () { return FILE ; } 

static int cwa_f ind_target_debug_f lag=0 ; 
iut sec_cwa_f ind_ targe t — debug (int x) 

{ 

int 6ld_f lag»cwa_f ind_target_debug_f lag ; 
cwa_f ind_target_debug_£lag=x ; 
return old_flag,- 

} 

typedef enum calib_orient 
{ 

CALIB_HORIZ*=0, 
CALIB_VERT 
} calib_orient ; 

/* A training parameters record for training rctatad rrcssha;r rr.cdels ♦ 



static ct 


r_params cwa_cai ib_ccrr_t? = 






{ 2. 


/* 


make a binary model 12 grey 1 


eve Is 




0. 


/* 


no leniency 






0, 


/* 


dcn~t measure angle 






5, 


/* 


default bias 




• 


1, 


/* 


left tail at IV 






1, 


/* 


right tail at IV 






50, 


/* 


threshold, not used fcr grey 


mcdel 3 




4, 4, 


/* mcdel 


resolution 






0, 


/• 


dcn't train for reader 






0. 


/• 


no angle training 






o t 


/• 


ditto 






0.0, 


/ * no s t 


andard deviation thresholding 


for stat 


istics 


128, 


/* care 








0 


/♦ 


NO flags set 







/ * (do not bother to initialized reserved fields; 



static cse_params 

cwa_calib_sp = {cse_absolute , cse_abs_binary , NO, NO, 1, 150 , 1000); 

/ * caliper used to find more accurate position estimates of edge 
* positions 
*/ 

static 

cclp_constraint cwa f ind_target_gcs [ 1 ) = 
{ 

{CCL?_CONTRAST, 0, 10 00, 0, 100. 3V, 
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/* scacic */ 

cclp_caliper clp_f ind_target_initial_search = ( 
LOOSE_CALIPER_WIDTH, /* Search Length */ 
LOOSE_CALIPER - PROJECTION_LENGTH. /* Projection Leng: 
2, /* Edge Filter Size 

0, /* Edge Filter Leniency */ 
10., /* Expected Size */ 

EDGE — THRESHOLD, /* Contrast threshold */ 

CCI*?_DONT - CARE, /* El Polarity */ 

CCI*P_NO_PAIRS , /* E2 Polarity */ 

CCI*P_DE FAULT , /♦ Optimization * / 

8, /♦ Bits per pixel */ 

TRUK , /♦ Window Rotation */ 

TRUE, /* Interpolate */ 

1, /* Number of contraints */' 
cwa_f ind_target_gcs, /* Constraints */ 
NULL, /* Pixel Map 

0, 0, 0 

>; 

/* static */ 

cclp_caiiper clp_f ind_target_f ine_search = ( 
TIGHT_CALI?ER_WI3TH, /* Search Length v 
TIGHT_CALI?ER_PROJECTICN_LENGTK, / * ?r3]ec:icr. Ler.cz 

2, /♦ Edge Filter Size 

0, /* Edge Filter Leniency -/ 
10., /* Expected Size * / 
EDGE_THRESHOLD , /* Contrast threshold 
CCLP_DONT_CARE. /* El Polarity */ 
CCLP_NO_PAIRS , /* E2 Polarity */ 
CCL?_DEFAULT . /* Opt im :a t icr. 

8, / * Bits per pixel * 

TRUE, / * Window Rctaticr. * 

TRUE, / * Interpolate */ 

1. / * Number of ccncra-nts * 
cwa_f ind_target_gcs , / * Constraints * / 
NULL. /* Pixel Map 

0, 0, 0 

); 



/* static */ 

cclp_caliper clp_f ind_target_compute_subpixel = 
TIGHT_CALIPER_WIDTH, /* Search Length +/ 

1. /* Projection Length */ 

2, /* Edge Filter Size 

0, / * Edge Filter Leniency 

10.. /* Expected Size 

EDGE_THRESHOLD , /* Contrast threshold 

CCLP_DONT_CARE , /* El Polarity 

CCLP_NO_PAIRS , /* E2 Polarity 

CCL?_DE FAULT . / * Optimization 

8, /* Bits per pixel •/ 

FALSE. /* Window Rotation */ 



95 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 



PCT/US97/18268 



FALSE, /♦ Interpolate 

1, /* Number of concraincs v 

cwa_f ind_target — gcs, /* Constraints */ 
NULL, /* Pixel Map */ 

0, 0. 0 

}; 

/* make an image of a rotated center of a crosshair, this image :s 
» used to train a model to search for the crosshair in ar. image 

* img - cip_buffer where rotated crosshair will be stored 

* orientation_in_tenths - orientation of rotated crosshair in tenths 

* of degrees 

* returns img 
*/ 

cip_buf f er *make_cross_hair_center_ image 
(cip_bu££er *img , int orientation_in_tenths ) 

{ 

cip_ buffer win. *xhair=NULL; 
cct_signal sig; 

NO_REG I STER { xha i r : ; 

if (:img) 

cct_error { CGEN_£RR_3 ADARG } ; 

if {sig=cct_cacch ;0) ) 
goto done; 

/* allocate a temporary cip_bufrer {xhair • which will :=r.:air. a 

* non-rotated crosshairs, and then use "his temporary =ip_buffer 

* as a source fcr crp_rctate centered 
*/ 

xhai r = cip_create ( img- >width*2 , img- >height*I , 3 ; 
cip_set (xhair, 0) ; 

cip_window (xhair, iwin, 0,0, xhair - >width. 2 , xhair- >heighr . 2 } ; 
cip_set (twin. 25 5) ; 

cip_window (xhair, iwin, xhair - > width, '2 , xhair- > height: / 2 , 

xhair- >width/2 , xhair- >height/2) ; 
cip_ set (&win. 255 J ; 

cip_rotate_centered (xhair , img, arier.cacicn_in_tenchsj ; 
done : 

if (xhair) cip_delete (xhair? , xhair=NULL; 
if (aig) cct_throw ( sig) ; 

return img ; 

} 

/ * construct a synthetic image and use search to find a =r:ss estimate 

* of the center position of the crosshair center 

* we expect that the synthetic model will be accurate tc 
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* approximately 1 pixel, and therefore, this search routine shculd 

* localize the center to approximately 1 pixel 

* irag - image containing a rotated crosshair 

* oriencation_in__tenths - orientation of rotated crcsshair in tenths 

* of degrees 

* res - pointer to cwa_point where result will be scored 

* gross_locate_crosshair_center_in_image () returns int signifying 

* whether or not it found a crosshair 
*/ 

static int gross_locate_crosshair_center_in_image 
(cip_buffer *img, int orientat ion_in_tenths . cvajjoint *res) 

{ 

cip_buffer ♦ targe t^NULL; 
cse_model •mdl » NULL; 
cct_signal sig; 
cse_results cse_res ; 
int ans; 

NO_REGISTER • target > ; 
NO_REGISTER{rr.di; ; 

if (sig=cct_catch CO) ) 
goto done; 

target = cip_create ( G RC S S _TARG E T _ S I ~ Z , 3?.CSS_TAF.GET_Si:*S . 3 :■ ; 

make_cross_hair_centsr_image (target, or:er.-aii2r._m_"er."hs ! ; 

mdl « ctr_train_model targe- . C. Z . 0, C , Z, 0. icwa_:al ib_=crr_tp 

cse_area_search '. img , mdl. icwa_calib_sc, icse_res'. ; 
if (cwa_f ir.d_target_debug_flag i Z :•: 1 . 
printf ( " Vd Vd Vd %d\n", 

cse_res . x, cse_res . y , cse_res . score , =se_res . re unci; ,- 

resox = 1 . * (GRCSS_TARGET_SIZE/2 - CIA_RNDI5 I cse_res . x . n : : 
res- >y = 1 . ♦ (GRCSS_TARGET_S I ZE .' 2 + CIA_RND16 i cse_res . y . r.j • ; 
done : 

if (target) cip_delete { target ) , target = NULL; 
if (mdl) ctr_deiete_mcdel (mdi ) , mdi=NULL; 
if (sig) cct_throw ( sig) ; 

return cse__res . found; 

} 

/* compute distance from a point to the closest peine on a line * / 
static double distance_f rom_point_to_line ( cwa_point *pl. cwa_line *li: 

{ 

cwa_line 12; 
cwa_point p2 .- 

12.X =• pl->X; 
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12. y = pl->y; 
12. t * ll->t + PI/2; 
cwa__lines_to__point (11, &12 , &p2) ; 
return (sqrt ( (pl->x-p2 .x) * (pi- >x-p2 . x) * 
(pl->y-p2.y) * {pl->y-p2 . y) ) ) ; 

} 

/* Fie a line to an array of points */ 
static double f it_line_to_points 
tcwa_point *pts # int n_points, cwa_line *line) 

{ 

int j ; 

double X3um=0.0, ysum»0.0, xcent, ycent; 
double a, b, c, theta, xp, yp; 
cwa_point *ptr; 

if (njpoints < 2) 

ccC_error (CWA_ERR_NO_ INTERSECT) ; 

for ( j=0, ptr = pes; j<n_ points; j — , per — 
if (cwa_f ir.d_target_debug_f lag i Oxl i 

printf ("Vf Yf\n'\ptr->x. ptr->y? .- 
xsura + = per- >x ; 
ysum += ptr - >y ; 

} 

xcent = xsum / n_poincs; 
ycent = ysum / n_points; 

for (j = 0, ptr = pes, a = b=c = 0.r j <r._pcint3 ; 1 — , per — 
xp = ptr->x - xcent; 
yp = ptr->y - ycent; 
a +o xp * xp ; 
b «•« 2 * xp * yp; 
c += yp * yp; 

} 

theta - a nan 2 (b , a-c) / 2.0; 

line->x - xcent; 
line->y » ycent ; 
*line->t ■ theta; 

if (cwa_find_target_debug_f lag & 0x1) ( 

printf ( " £it_line_to_points returned : *f - , 
(a*cos (theta+PI/2) *cos (theta+PI/2! + 
b*cos (theta+PI/2) * sin (theta+PI/2) + 
c*sin(theta+PI/2) *sin (theta+PI/2) i /n _points) 
cd_showas ( ( int) line , "cwa_line" ) ; 

) 

return ( (a*cos (theta + PI/2) *cos ( theta+PI/2 : * 
b*cos (theta+PI/2) -sin ( theta+PI/2 ) - 
c*sin(theta + PI/2) *sin { theta+PI/2) '; ,'n ^points; ; 
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static int print_list_o£_points 
(cwa __poiat *val, int num_vals> 

{ 

iat i; 

for (i a 0; i < num_vals ; i**) 

printf ("%f %f\n", valfij .x,val(ij . y) ; 

) 

/* calib_orient_f rom_angle () returns CALIB_VERT or CALIS__HORIZ 

* depending upon the orientation (measured in degrees) of a line, Its 

* purpose is to determine whether we need to use a vertical or 

* horizontal quadratic subpixel edge interpolation 
*/ 

static calib_orient calib_orient_from_angle (double angle_m_degrees i 
double angle_in_rad; 

angle_in_rad = angle_in_degrees * ? I / 180 . ,- 

if (fabs (cos (angle_in_rad; . > 

fabs (sin (angie_in_radi : 

return CALZ3_VZRT; 
return CALX 3 HCRI3 ; 



/ * computes subpixel edge pcsmcr.s given ar. image and a r.eighicrhccc 

* ( ap_x ( ap_y) , to search for ar. edge. Depending upon the 

* orientaticn_in_degrees , ccrr.pute _subpixel pes it icn ( ) will either 

* perform a vertical or horizontal subpixel edge es:ina::cr., Subpixel 

* edge positions are computed ir. the same manner as the boundary 

* tracker (using 7 neighboring pixels, and 7 differences where 

* we quadrat: i rally interpolate the edge position from at least three 

* differences. 

* We compute a seven first differences just to he safe. There is no 

* guarantiee that the computed caliper position corresponds to the maximum 

* 1st difference (because we're using calipers with filter size 2) 

* img - cip_buffer containing image of crosshair target 

* ap_x , ap_y - application point where we sample image grey values 

* actually, we sample grey values at the application 

* point and ♦ 2 from the application point (maybe we 

* should sample at -»-/- 4, who knows 

* orientation_in_degrees - orientation of caliper which has been applied 

* to find this application point. Since we 

* only want to call subpixel interpolation 

* at the right place, we suggest using 

* on-axis calipers 

* pos_x, pos_y - pointers to doubles where 

* compute_subpixel_position i ) will store result 
*/ 

int compute_subpixei_position 
(cip_buffer *img, int ap_x. int ap_y, 

double orientation_in_degrees . double ♦pos_x, double *pos_y) 
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inc pO, pi , p2 , p3 . p4 , pS , p6 , p? ; 

int dO.dl, d2,d3.d4,d5,d6, d_max, d_mm; 

calib_orienc direction ; 

if (cwa_f ind_target_debug_f lag & OxIJ 

print f ( "compute_subpixe 1 _pos icier, called wi:h Vd Vd *f tx *x 
ap_x, ap_y, orientaticn_in_degrees . pcs_x , pcs_y * ; 

if (ap_x < 4 | | 

ap_x > img->width-4 | | 
ap_y < 4 | | 

ap_y > img- > height -4 ) { 
return 0; 

> 

/* should we sample horizontally or vertically 

direction «= calib_orient_f rcm_angle {orier.tat i=r._i-_degrees 

if (direction == CA^I3_HCRIZ: { 

p0 = * ( img - > rat Jap_y - ] *img- >x_c::ssc-a; x. 
pl = * ( img- > rat iap_y- 3 ] ritng- >xcf f set *ap_:c 
p2 = * (img - > rat Jap_y- 2 • *img- >x_cf f set-ap_:<; 
p3 = * ( img- >rat {ap_y- 1] -img- >x_of f sez ^ap_x. 
p4 = » ( img - >rat Eap_y) -img - >xcf f se: ~ap_x; .- 
p5 = ♦ ( img- > rat [ap_y I ) -img - >x_cf f set-ap_x: 
p6 ~ * ( img - > rat (ap_y-2 ] *-img- >x_of f se"*ap_x; 
p7 = * ( img- >rat (ap_y-0 j - img - >x_of f set +ao_x', 

d0«pL-p0 
dl=p2-pl 
d2=«p3-p2 
d3 =p4 -p3 
d4=p5-p4 
d5=»p6-p5 
d6 =p7 -p6 

d_max = max (do , max idl , max (d2 , max i d3 , max (d4 , max '.d5 , d6 ) ■ ■ ;- 5 ; 
d_min a min (do , min (dl , mir. ( d2 , min ;d3 , mir. (d4 , min(dS . d6 ; ) \ ] ) ) 



if 



) 



(-d_min > d_max) 
d0=-d0; 
dl=-dl 
d2=-d2 
d3=-d3 
d4=-d4 
dS=-d5 
d6=-d6 
d max =-d min; 



*pos_x = ap_x*l. - 0.5; 

/ * If we're at a maximum return "he subpixel position 
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if 



td_max > SBGS_THRESKOLD) { 
if (dl « d_raax && d3 ! = dmax ) { 

*pos_y - ap_y * c = _jjarf it idO . dl . d2 ) .'6 5536 . - : 
return 1; 

) else if (d2 = = d_max && d4 '= d_maxj { 

*pos_y = ap_y + cz _parf it Idl , d2 . d3 ) ,'6 5 5 36 . - 1 
return 1; 

} else if (d3 =» = d_max && d5 ; = d_max) ( 

*pos_y = ap_y + cz _parf it (d2, d37d4) /63S36 . ; 
return l; 

} else if (d4 « d_max && d6 != d_max) { 

*POS_y » ap_y + cz_parf it (d3 , d4 , d5) /6S536 . +1 
return 1; 

} else if (d5 == d_max) { 

*P os _y - a P_Y + C3_parfit(d4,d5,d6J/65536.»: 
return 1; 

} 

/* If we're not at a maximum , retur- 3 s~me- u 
*/ 

return 0; 



} 

} else 
p0 = * 
pi = * 
p2=** 
p3 = * 
p4«»* 
pS = * 
p6 = * 
D7«* 



{ 



(img- >rat ;ap_yj -img- >x_of f set ~ac_x - 4 : 
( img- > rat [ap_y] ♦img - >x_cf f set -ac x - 2 ' 
(img->rat tap_yj -img- >x_o£fset ~ap~x- 1 : 
{ img- >rat [ap_y ) ~img- >x_of f set -ap~x- i; 
(img- >rat [ap_y] *img- >x_cf f se: -ap_x: . 
( img- >rat Cap_yj -irr.g- ^x_of f sec-ap x-1! 
v img - >rat Jap_y! *- img - >x_of f set -ap~x-» : 
(img- >rat [ap_y) -img- >x_of f set »aD x-3 : 



dOspl-pO ; 
dl=p2-pl; 
d2=p3-p2 ; 
d3»p4-p3 ; 
d4=pS-p4 • 
dS=p6-pS; 
d6=p7-p6 ; 

d_max = majc(dO,max(dl.max(d2,tTvaxid3.max(d4,max(dS.d6) ) } } ; ) 
d_min = min (do , min idl , mm ( d2 . min (d3 , min (d4 , min (ds . d6 ; })) ) ) 

if (-d_min > d_maxi { 
dO=-d0; 
dl=-dl ; 
d2=-d2 
d3=-d3 
d4=-d4 
d5=-d5 
d6=-d6 

d_max = - d_min ; 

} 



/* If we're ac a maximum, compute the subpixei oosizicn and 
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* return i 
•/ 

*pos_y = ap_y * i . + 0.5; 

if (d_raax > ZDGE_THRESHOLD> { 

if (dl » d_max ii d3 I = d_max) { 

*pos_x = ap_x*l . + cz_parf it (do , dl . d2 ) /6 5 536 . - 2 • 
return 1; 

.} 

else if (d2 -» d_max fc& d4 d_max) { 

*pos_x = ap_x*l. + c =- parfic(dl ( d2.d3)/65536.-i. ; 
return l; 

} 

else if (d3 d__max dS ! = d_max. 1 { 

*pos_x = ap_x*l. + cz_parfit (d2.d3.d4}/65536. ; 
return 1; 

) 

else if (d4 == d_max && de 1= djnaxi { 

*pos_x = ap_x»l. + ==_?arfi- ;d3,d4.d5- . 55 = 35. -1. ■ 
return l; 

} 

else if ids == d_max: { 

*pos_x = ap_x-l. - cz_parfir '.d.;,d5.d5. /5553S. -2. . 
return I; 

} 

/• If we're not at a maxir.u-. scme:h:r.c:'s wren- a-^ — t 

*/ " — « 

return 0; 

} 

} 



enumerate^oints_and_fit_iine_aIcr.g_ray<) enumerates edge saints 
along a ray from a start_pcint cc an end_coint bv usinc calipers 
(and possibly subpixei interpolation; co comoute" edge positions 
enumerate_point 3 _and_fit_line_along_rayt) continues enumerating 
points until one of the following conditions occurs: 

0 . we reach the end_point 

1. no edge is found by the caliper 

2. more than one edge is found by the caliper and the highest 
scoring edge is less than twice the score of the second highest 
scoring edge (this 2X comes from the CONFUS I0N_THRESHOLD ) 

3- the computed edge point is an outlier {the distance between the 

edge point and the nominal line is larger than threshold) 
4. we run off the cipbuffer (screen) 

After it has enumerated the edge points, it then calls 
f it_ooints_to_line to compute the optimal line estimate 

img - cip_buffer containing a crosshair target 
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start: _poinc . end_point - define region along which :c acolv calicos 
nominal_lme . chreshold - used to determine whether an ecae ocinc 

is an outlier {and consequently end the 
enumeration of edge points) . The outlier 
test involves checking whether the 
distance from the point to the 
step_size - frequency of edge point sampling. Since we want 

evenly sample the lines, we move step_size number of 
pels in the larger direction (and a fractional number 
of pels in the other direction) . This approach provides 
even sampling assuming we are doing on-axes caliper ir.g 
or parabolic subpixel interpolation 
search_caliper - caliper used to search for edge points 
computed_line - cwa_line where result will be returned 
enumerated_points - optional array of cwa_points where found edge 

points would be stored 
num_jJoints - pointer to an int where the number of fcur.d ccmts 
would be stored 

do_subpixel_interp - flag characterizing whether we want tc cerfcrrr. 

subpixel ir-terpclaticr. after app lyir.r rr.s calicer 

static int enumera te_points_and tic line alcnc rav 
(cip_buffer *img, cwa_pcint *start_point7 cwajc;:.: *enc ccir.t. 
cwa_line ♦nominal_line . double threshold, double step_sice. 
cclp_caliper *search_caliper , cwalir.e *ccmputed_lme . :va_pc:r.: 
*enumerated_points, int *num_poir.cs . int dc_subpixel ir.terc 

int i , j , k ; 

double maxAbsCosSin. 

cwa point ap ooint , *enum ccmt. vec; 
int ap_x, ap_y ; 

ccip_resuics r e s_h ; N_EDGES * 2 - 1 ! . 

cclpjparams cpp = { SEGE_/TK?.Z3KCL2 * 1 3 , N_£2GE3 . 2 • : 

double cclp_angle 

int num_sampl e s , tmp ; 

double caliper_vert ; 

cct_signal sig; 

cwa_line sampling_line ; 

if <cwa_f ind_target_debug_f lag & Oxl) { 

printf ( " enumerated j)oims_and_f it_line_alcng_ray called with .n"; 

printf ("img Vx start_point Vf Vf end_point Vf vf \n" , 

img, start_point- >x, start_point - >y. end_ point - >x. er.djici.ic - >y) ; 
printf ("nominal_line %f Vf t Vf threshold Vf \n" , 

nominal line- >x, nominal line- >v . nominal line->t "~ -°shold < • 
) " 

/* Compute the sampling line */ 
sampling_line .x = start__poinc->x; 
sampling_line . y = start_point- >y ; 

sampling_line . t = atan2 (end_pcint - >y- start — point - >y . 

end_point - >x- starr_pcir.c - >x) ; 
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/* Compucs che seep which is integral in x cr v — c----a-^s 
*/ 

if (fabs (cos (sampling_line. c} ; > f abs i sin t sampi.Lr:g_l:.ne . c . 

ntaxAbsCosSin * fabs ices ( sampli-g_ii."e . c :• * ; 
else 

maxAbsCosSin = fabs ( sin ( samplingime . c) ; ; ' 

vec.x = (cos ( samp ling__ line, t) /maxAbsCosSin J * seep size; 
f vec.y » <sin[sarnpling_line. t) /maxAbsCosSin) * step_si=e.- 

/* Compute Che number of samples which we will be makina 
•/ 

num_samples = 

max(abs( tint) (starc_poinc - >x - end_pciat->x) 5 

abs ( (inn) (start_point - >y - endjjom: - >y? , 
step_size; 

if ( f enumerated points ) 

enumerated_pcints = 

(cwa_pcinc * ) chp_ailoca { nu-_sarr.pl es 'sicec z ; cwa_:::.-.: 

if ( ! num_points; 

num_poincs = irmp ; 

cclp_angle = samplmg_i:r.e -~~130..'?:-9C.; 

if (do_subpixei_interpi { 

if <calib_orient._from_angle-.ccip_ar.gle: =~ CA1.I3 VERT : 

cclp_angie =0; ~~ 
else cclp_angie = 90; 

} 

if (cwa_f ind_target_debug_f lac & Oxl; 
print f ("ccip_angie Vf \r. , cclp_angie ; ; 



Inner Loop: Enumerate edge points 



for { i - o, ap_point.x = scar- jsoinc - >x , . apjpcmc . y = 
start _point->y, enum_point ~ enumerated_poincs , 
*nura_points =» 0 ; 
i < num_samples ; 

i-t--^, ap_point.x + = vec.x, ap_point.y *■= vec.y. er.unijjoir.t**, 
(*num_points) +->-) { 

ap_x = (int) (ap_point.x * 0.5) ; 
ap_y = (int) iap_point.y * o.S); 

if (cwa_f ind_target_debug_f lag i Oxl j 

printf J "ap_x \d ap_y \d\n" , ap_x, ap_y: ; 
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res_h[0] .found » res_h(lj .found = g. 

/* cip_cransform throws CGEN ERR BADARG c^P pop or- a — » 
I ^W_ FI r when caliper^^n exLdi I sf""??^ • - 
Che cip_buffer. We wane to break off the sea--' a-Tr""* " 

* point anyways. so we catch the e-cr anrt a : — s 

* line to the points ^ 3Ump CC = — a 
*/ 

if (sig = Cct__catch(CGEN_ERR_BADARG) i 
go Co fit__line; 

if (sig s cct_catch<CIP_ERR PEUUDDR)) 
goto f inline ; 

if (sig = cct_catch(CCLP Err FIT j , 
goto fit__line ; ~ ~ 

cclp^apply 

(3earch_=aliper, img.ap_x. ap_v. 3r= ^ . 

cct_end (CCL?_ERR_7;-r; ; 
cct_end(CX? - ERR_ ? ELACDR; ; 
cct_end(CGEN_ERR_3ADARG; ■ 

/• check if edge is not found * 
if ( !res_h[0] .found) 
goto £it_line; 

/* check if mere than cr- . - - ^ 

if (res_hri S . foussd i& ^ ^= , . 

re» hfol acore c CCNrTJS ICN THRESHOLD* res h'M 
if (cwa_f^d_carget_debug_fla g fc 0x - / 

cd_showas <res_h, "cclp_resul ts " • ■ 
^ cd_showas(ires_hi;} . "c::l?_results" . ; 

goto fit liner 

} 

if (cwa_find_target_debug_fiag & 0x1) f 

printf(-ap_x %d ap_y %d angle xf < os ^ m 

*e / /!-?' ap - y ' CClp - an 5 le ' res -hE0] .aositicn) • 
if (fab 3 (res_h[0] -position) > 

cd_showas (res_h, "cclp_results" ) • 

cd_showas (&res_h(i] , " cclp_results" } ; 

} 

/* compute the subpixel point measured by the caliper - ■ 
enumjpoint - >x = 

e nu^ci„;.! y S ; reS - h( ° ! ip_an gle . ?I/ , 90; ; 
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if tdo_suboixel__incerp) { 

- * { .* =3mpute_subpixei_posi- - ~ 

goto f inline; - olac >x V *^umj>o irit: _ >y , 
if (cwa_find_cargec debug fi aq & 3afl| 

#endif 

if (di.scance_frcm_poi n r — 

threshold) ~ - er -- ffi _?cir.= . r.cm— lne • > 

goco fit i iSa . 



/ 



/' Sic a liae - 3 -_ he pcia _, _ 

any oad points -or- . n " — e-:cur.:a- 2 « 

• /" ' - ~ - "ervise ~ ~ 

f i t_l iae_ t o_points 
static ia - rou,d_nu^ er: ^ ie x; 



> 



return { 
ise 

return tint) (x-o.5) ; 



return <i nC ] (x^o s> • 
else 



double diff,res ; ~*" 
int modDif f ,■ 

Pn.-.( avg *f tf orienc.i. 3r i«, e _ 2 , . 

- ori.„ c _i; 

™ = rouna_number (ci- " a- - . 

-es= ; (ori«ne wl ,orienc_2-niodiif5»Pr;. .-2> , 



106 



SUBSTITUTE SHEFT mm c m 



WO 98/18117 

PCT/US97/18268 

" (cwa - fi " d »=argec_debug flag i ^ ■ 



prxncf £- = %ff\n".res> - 
^ re cum res , 



/ V COIBPUCe a " 9le coo-^nace, gi, e , bv :ha _ g _ 

(pr,v_-. ransfora . Mpi 5ouble .^^^^--P 
double realSin.reaXCos; 

} r a cura(acan2 (realsin, realCos, . ? ' 

/* average che orientations =5 ihe r 

composite orientation estira-e ",~ /S " C bes-. 

.rr «•*• » y , shouirbl o::::^;^ 3 ;; 3 -------"--- 3 =-* --=« 

; — .cac.ons are aligned vit- " a — n —-sr. rhe 

characterized a her a - - — . - ; 3C if . _ . . , . 

I by ?:.-, =3 bec^ e "h:::*::J; en ?~- = ; -- hs ^ y be" 

orientations . *' — sn we average rhe resu' 

sta::: double average ' c »- 

{ " -a._o. d c ^ ie --g crier.: 



double phvs orie~- ' * ' 

"*-* av 9_-^g_orier.r ; 

F - rs -' -ansf^m ancles <= , 

* coordinates " —age -.corciances tic =hv S '- a - 

*/ 

for (i = 0; i < 4; 
phys_orienc [i] = 

transform^angle according - 0 

H(pr:v cwa cailb • ; ^-kT -°-- a - ibr «-°n_:nap 

- ' - a -«^b; - >map. itng_ or ;enc fii' ; 

if (cwa_find_tar g ec_d e buc_fiag * 0 xij 
princf (-Vf\n».pny S _ =rient :i; ♦ . 

■'• Shift orthogonal angles by PZ/2 ir o rde . r „ - - 

-*rst orientation ~ -° i-ne up with «"he 

*/ 

for <i - 1; i < 4; 

if <fabs< s i R{ p hys ^ orieRz: c~- er -r 0 ,.. 

phys_orient (i) ? :/2 . " ' ~ r -~'°- ■' >sqrt . : . s ; : 
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/* average Che orientations • 
avg_phys_orient = 

a ve rage_ two_o r i enc a t ions 

(average_two_orientacicns ;phys_cr ler.c ;c • . phys_crier.r ; , 
average_two_orientations ;phys_orienc [2] , phys orient ^3; ! } 

/* compute the orientation in image coordinates corresponding 

* the averaged orientation 
. */ 

avg_img_orient » 

transf orTn_angle_accarding_to_caiibration map 

{ ( (priv_cwa_calib * ) calib) - >inv_map, avg_ohys_orient: ; 

if (cwa_f ind_target_debug_f lag £> 0x1) 

printf { "phys *f img tf \n H , avgjphys_orienc , avg img_cnent) ; 

return avg_img_orient : 



} 



/* compute the image or.e-:a::cr. for wr.icr. -he physical ;r:ar.ia::c: 
* is orthogonal to another imaae orientation 
•/ 

static double image_r.crmal_^c_ image _angl^ 
(cwa_calib * calib, double img_crienc 
{ 

return 

( trans f crm_angle_ac=crdir.g_*tc_calibratior._map 
( ( (priv_ova_calib ' ! calib. ->mv_map, 
( trans fcrm_angIe_acccrdmg_co_calibracicr._:nap 
( [ (priv_cwa_calib * .' calib. - >rr.ap , img_-r ler.r ; - PZ.Z) '. ) ; 



* compute_center_positicr._and_angie_assuming_:cur_guaranteed good rays i > 

* improve estimate of the crosshair position and orientation by 

* searching four regions of guaranteed good length (50 pixels) 

* eminating from the crosshair center. Lines are fit to the edge 

* points along the four regions, and then the orientations of these 

* lines are * averaged" . Two orthogonal lines are computed by 

* averaging pairs of the four rays Finally, che position estimate is 

* computed by intersecting the two orthogonal lines 

* img - cip_buffer containing an image of a crosshair target 

* calib - cwa_calib object characterizing pixel non- rectangular ity 

* estimated_ctr_img - estimated position (given in image 

* coordinates) of crosshair target i computed in step 

* estimated_orientation_phys - user- supplied orientation estimate 

* (given in physical coordinates) 

* guaranteed_good_di stance - length {in pels) of guarateed good 

* region extending outward from che origin 

* computed_center - structure where computed position will be stored 
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* computed_orientaticn_phys - double where computed crier.ia: in 

* physical coordinates) will be stored 

* good_rays - array of four cwa_line structures where good rays 

* {lines fit to sampled edge points m the gccd rsgicrj 

* will be scored 
*/ 

static int compute^ center_posi t icn_and_angle_assuming_f our_guaranteed_gcod rays 
(cip_buf f er *img, cwa_calib *calib . cwa_ point *escimated_ccr_img. double ~~ 
estimated_orientation_phys . double num_guaranteed_goad_pomcs , 
cwa__point *computed_center , double *computed_orientation_phys , 
cwa line *good_ray3) 

{ 

int: i; 

double angle [4 ]; 

cwa_point start jpoint (4 ] , end_poinc (4 ] ; 
cwa — line computed_line [4 ) , ncminai_lme ; 
double . img_angie f 4 ) , angle_horiz_iine ; 

angle [0] = es timated_or lent at i cn_phys ; 

angle [1] = estimaced_orientaticn_phys~?:: .' Z ; 

angle [ 2 ] = estimated_cr 15nrac1cr._j3h.ys »PI 

angle (3] = esc:maied_srisr.^azicr._phys-?: * - . E .- 

/* We define the 50 pel GUARANTEE!: GCCD REGION 1- terr.s cf a 

* VALID REGION so that we can use 

* Che enumerate_points_and_f it_line_alor.g_ray * ; function 

* If we're sampling along a line and we wane to seep sampling 

* after 50 pels, then we can use a nominal line which is ncrrr.al 

* to and passes through the stare point, and a threshold of 5C 
*/ 

nommal_line . x = estinated_ctr_img- >x; 
nominal_line . y = estiniated_ccr_img - >y ; 

/ * The passed orientation is given in physical coordinates 
for (i s 0; i c 4; l-r~) 

angle [i] = transf orm_angie_according_to_calibratior._map 
( ( (priv_cwa_calib *) calibi - >inv_map, angle [L; ) ; 

for (i o 0; i < 4; i*+J { 

/* The sampling region begins at the poinc a small distance 

* away from che crosshair center 

* (CIRCULAR_SUFFER_ZCNE_AROUND_CENTERi to be exact, and 

* excends out to guaranteed_good_distance from the center 
*/ 

start_point [i] -X - escimated_ctr_img- >x + 

CIRCTJLAR_30TFER_Z0NE_AROUND_CENTER*cos (angle I ij • ; 
. scart_point [i] .y - escimaced_ccr_img- >y + 

CIRCULAR_BUFFER_2CNE_AROUND_CENTER* sin ( angle [ i ] } ; 
end_point [i] . x = estimated_ctr_img- >x + 

num_guaranteed_gcod_ooints*cos (angle [i; i ; 
end_point [ i] . y = estimaced_ctr_img- >y * 
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num_guaranteed_gccd_poincs » s ih ( angle ; i ; : 
nominal _ 1 ine . c = angle t i J ♦PI/2 ; 

enume ra ce_ po in t a _and_f i t _ 1 ine_a 1 o ng_ ray 

(img, 6scart_point (i] . &end_poinc fij . &ncminal_lir.e . 
nun_gruaranteed_good_points . 

STEP_SIZE_USED_T0_ENUMERATE_?OINTS_IN_GCOD_RZ:3IO?I*2 . . 
&clp_f ind_target_inicial_search, 
&computed_l ine [ i ] , NULL . NULL . 0 ) ; 
itng_angle £ i 1 = com puted_line Ci ] . t ; 

cu copy (&campute d_ line til » &good_rays [i] , sizeof (cwa_line) ) ; 

/* average Che angles */ 

angle_horiz_line = 

average_four_image_oriencacions t'calib, img_ar.gie'. .- 

if (cwa_£ind_target_debug_f lag & 0 x I ; 
printf ("average Yf Vf Yf Vf %£'n M . 

img_angie { 0] , img_ar.gle ' 1 ; , ir.g_ar.gle \ Z * . iT.g_ar.gle \ 2 * 
angle_hori=_line) ; 



average che fir: rays 



compuc ed_l ine 1 0 ) . x 
computed_line [0] .y 
computed_line [l] .x 
ccmpuced_line [1] .y 



( ccmpuced_l ir.e [01 . x - corapurec_ 
{ compuced_iine CO] .y corapuced_ 
( computed^! ir.e [ I ) . x - computed_ 
{compuced_lir.eC!! '/ - com puted_ 



ne [2] .y: .': 
r.e [ 2 \ ,x!/; 
r.e .3 ■ .y:.". 

ccmpuced_line 1 1! . t = image_nsr— al_tc_i~ace_ar.cle calib . ar.gie_hcri t_lir.e ! 



ccmputed_line {0} . t = angie_hcri = _lir.e ; 



/* compute che :.i:ersec:ed pc ir.z *. 

cwa_l mes_co_pcint ( &cc-puted_l ine [C] , iccmpu-ed_l ir.e [L] , cc~puted_cer_cer .* ; 

/* fix che orientations of the good rays to correspond to the 
* "average" orientation 
*/ 

good_rays to] . t = good_rays 1 2 ) . t = angle_horiz_line ; 



*computed_orientation_phys = 

transf orm_angle_according_to_calibrat ion_map 

( ( (priv_cwa_calib *)calib) - >map , angle_horit_line) ; 
good_rays[l] .t = good_rays[3] .t = 

trans form_angle_according_to_calibraticn_map 

{ ( (priv_cwa_calib * ) calib) - > ir.v_map , 
*compuced_orientacion_physt-PI/t» ) ; 



Compute the two intersections between a circle and a line if the 
circle and line intersect. If the circle and line do not 
intersection, signal C^A_ER-R_NC_ INTERSECTION 

/ 
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scatic inc inters ect_iine_and_circle !cwa_l:ne *lir.e, c^a _pc:r.: *? _ . 

double radius. cwa_poinc *res_l. 
cwa_po int *res_2J 

{ 

double angle_co_line, discance_:o_line, angle_to_pcir.c_cr._lir.e : 

angle_Co_line = line- >t+PI/2 ; 
if t (cos (angle_to_line) • (line- >x-pc- >x) + 
sin (angle_to_line) * (line- >y-pt->y) ) < Oi 
angle_co_line >=• PI; 
distance_to_line - discance_f rom_point_to_line (pt, line) ; 

if (distance_co_line > (radius - EPSILON_THRESHOLD) ; 
cct_error (CWA_ERR - NO_INTERSECT) ; 

angle_to_point_on_ line = acos (distance_tc_line/radius ; ; 



r.e: ; / 



/* compuce_center_posit_on_and_ar-gie_alang_g"uaranteed_gocd_rays '. *• 

* generate final estimate of crosshair position and orier.:a::cr. by 

* enumerating points along four rays (using four good rays as 

* thresholding lines) . We use tighter calipers because we trust the 

* positions of the good rays. 

* img - cip_buffer containing an image of a crosshair- target 

* calib - cwa_caiib object characterizing pixel non- rectar.guiari ty 

* est imat ed_ctr_img - estimated position (given in image 

* coordinates) of crosshair target (computed in 

* step 2) 

* good rays - array of four cwa_line structures where good rays 

* (lines fit to sampled edge points in the good region) 

* will be stored 

* tight_threshold - threshold used for determining outlier points 

* with respect to the good rays 

* computed center - structure where computed position will be stored 

* computed orientation_phys - double where computed orientation (in 

* physical coordinates) will be stored 
*/ 

scatic int compute_center_position_and_angle_along_guaranteed_good_rays 
(cip_buffer *img, cwa_calib *calib, cva_poxnt *est imated_ctr_img , 
cwa_line *good_rays , double t ight_threshold , cwa_point 
♦computed center, double *ccmputed_orientaticn_phys , 
double * feature score) 



res_l->x = pc->x + 

radius *cos ( angl e_to_l ine +angl e_t o_pc int _on_l jl 
res_l-»y = pt - >y + 

radius *s in ;ar.gle_to_lir.e-rar.gle_to_ ?c:r.:_or._l; 
res_2 - >x = pt->x + 

radius*ccs ; angie_to_l-ne-ar.gle_to_pcir.t_or._li 
res_2->y = pt - >y * 

radius * s in { ang 1 e_t c_ 1 ir.e - ar.g I e _t o_pc in t _or._l i 
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int i; 

double angle [4 J, - 

cwajpoinc start_point (4 } , end_point £4) ; 
cwa_line computed_line [4 ] ; 
double img_angle (4 J , angle_horiz_iine ; 
inc num^enumeraced^ points [4 J ; 

cwa_point poasible_start_points (2] . enumeratsd_poinrs f4 J ;MAX_NUM_ POINTS ] 
double avg_squared_error ; 

if (cwa_f ind_target_debug_f lag & 0x1) 

printf t "compute^ center^ position_and_ angle_alcng_guaranteed_good_rays\n 

for (i _ 0; i < 4; f 

double distance_co_center , radius , orien:at;on_away_from_:en:er ; 

/+ For each ray, we need to dec ermine whac che sampling 

* line should be 

* We do this by intersecting che ray wich a circle 

* centered at the crosshair center 

* Since there are twc ir.cersecc icr. ?o:r.:3. we need cc 

* dec ermine which ere is the real s care z t che sarrxlir.c 

* line 

* It turns out that we car. sirr.ply picit che ir.ee rseccicn 

* point which is closer tc che center zt che ray 
*/ 

distance_to_cencer =* 

distance_f rom_jjoir.t_co_line -:escimated_ccr_img, igood_rays [i] ; ; 

radius = sqrc ( C I R CUI-AR_3 r ER _ 2 ONE _ AAG UND_C ENTE R * 
CIRCUI*AR_3Ur"rER_2CrTE_ARCU>IC_CZ:rr£?.- 
discance_co_cencer*distar.r e_co_cer.ee r": ; 

intersecc_iine_and_circie ( £good_rays [L\ , estinated_ccr_img , 

radius. ipossibie_start jpcints [0] , 
_possibie_scarc_ccir.t:s [lj : 
if (cwa_point_distance 

(fi t possible_start_points [0\ . (cwa_point * • igcod_rays (i] 1 < 
cwa _po in t _d i s t anc e 

(&possible_start_points [1] , (cwa_point *) igood_rays [i] ) ) 
cu_copy ( tpossible_scarcj?oints CO] , iscart_point til » 
sizeof ( cwa_poinc ; } ; 

else 

cu_copy ( tpossible^starcjjoints [1; , &start_point l1] , 
sizeof (cwa_pcir.c! ) ; 

/* Now, we have to determine which way we should travel along 

* the ray (good_ray [i] . t or good_ray [ i ] . t +?I 

* We determine which way by comparing che start point with 

* the center of the geed ray 
*/ 

orientacion_away_£rom_center = good_rays ( i • . z ; 
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if ( (cos (oriencacion_away_ f rom_centeri * 

{good_rays [ il . x-escimated_ctr_img- >x, - 
sin (orientation_away__f rom_centeri * 
(good_rays [i] . y-escimated_ctr_img- >y; . < C : 
oriencation_away_from_center *o PI ; 

end_point(il .x » 3tart_point [i) .x * 

MAX_LENGTH_D I AGONAI*_ACROS S _C I ?_BUF FSR * 

cos <oriencation_away_from_cencer) ; 
end__point [il .y » start_point fil . y + 

MRX_LENGTH_D I AGONAL_ ACROS S _C I P_BUFFER * 

sin (orientation^ away_f rora_center) ; 

if {cwa_f ind_target_debug_f lag & Oxl) { 

princf ( "center Vf Vf good_ray Vf Vf Vf distance Vf* 

estimaced__ctr_img- >x , escimaced_crr_img- ;>y . 

good_rays fil .x, gocd_rays £ i ] . y , gccd_rays * i ; . - . 

distance_to_center) ; 
princf i M Vf Vf Vf Vf \a" . 

starcjscinc;:; . x . s:art_scm: [ i ; .y. 

end_pc;n: [ i ' . x , enc!_pcir.^ [-'. ■ y ; 

) 



enume r at e jjo ini 3 _and_ f i c_i me_along_ray 

(img, &start_pcinc [i] , &end_pcint [ i ] . &gccc_rays \ i \ , 
tight_threshold, 

STE?_S X2E_USED_TO_ENCMERATE_ALL_?C INTS . 
&clp_f ind_target_f ine_search . 
/ *&cip_f ind_target_ccmpute_subpi:<el , * 
&computed_lire [ i ; , &er.umeraced_pcir 4 :s : [0] . 
tnum_enumera"edj;oi"t:s ' i ; , 1; 
img_angle (i ; « ccmDUted_lir.e { i ] . 1 ; 

} 

/* For each pair of opposite rays, use the closest points sc that 

* we use the same number of points on both sides of the 

* crosshair center 

* Since we don't know why the last edge point became invalid, we 

* skip the neighboring 2 points just to be on the safe side 
♦/ 

num_enumerated_points [0] = num_enumerated_points [2 J = 

min(num_enumeraced_poincs CO] , num_enumerated_points [2\ '. -CUTLIER_3UFFER ; 
num_snumeratedj3oincs [Lj = num_enumeratedjpoints [ 3 • = 

min (num_enumerated_points (1] , num_enumerated_?oints [1] :■ -CUTLIER_3UFFER ; 

/ * Re -fit lines to these points 

avg_squared__error = 0 ; 

for (i = 0; i < 4; i+-) { 
double fit error; 
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} 



fic_error » 

f i t_ 1 ine_to_po ints 

(&enumerated_points fx] [0} . num_enumeratad_pair.:3 . 
&computed_line (ij ) ; 
avg_squared_error +-= f ic^error/4 ; 
img_angle[i] = computed_line { i] . t ; 



*feature_3core = 

C2_map_average_squared_error__co_max_1000 (avg^squared_error) ; 

/* Average the four rays by averaging the positions and 
* orientations 




computed_line [0] .x 
computed_line £01 .y 
computed_line [1] .x 
computed_iine [1] .y 



angle_hor iz_l ine = 

average_fcur_image_oriencacior.s • cal . :r.c_ar. 
computed_line [0] . t = angie - hor_=_line ; 
computed_line [1] . t = image_norr!tal_tc_image_ar.g 



ineC-I .xJ 
ine;2j .yj 
ine ■ .x; 



cwa lines_co_point < &ccmputed_iine [0] . tcompuced_i:ne '.'. \ . ccnpu:sd_csr.:er "■ ; 

*computed_orientation_phys = 

trans form_angle_ac=3rdir.g_cc_caIibratrcn_niap 

t ( (priv cwa calib * ) caiibt - >tnap, ar.gie_hori=_lir.e> ; 



* cwa_f ind_targec ! .) is the top level function for localizing 

* the crosshair target. It returns C if it cannot find the crosshair. 

* 1) use search to improve position estimate of crosshair center 
» 2) compute four good rays eminatir.g from the origm by sampling 

* edge points within the guaranteed good region along the user 

* specified orientation 

* 3) Enumerate subpixel edge points along the four rays eminating 

* from the the origin and use the found good rays to identify outliers 

* cwa_f ind_target tries to localize a crosshair target within an 

* image and returns 1 if it was successful and 0 otherwise 

* img - cip_buffer containing crosshair target 

* calib - characterizing field of view 

* angle - initial orientation estimate of crosshair target 

* result - pointer to cwa_line structure where result will be stored 
*/ 

int cwa_f ind_targec_ internal 

(cip buffer *img, cwa_calib *calib. double angle, cwaline -result . 
cwa status * status) 
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cwa_point grossjjoa^escitnate, improved_pos_estimate ; 
double improved_phys_orientat ion_estimate .- 
cwa_line good_rays {4 1 ; 
iac i , ana ; 
ccc_signal sig; 
ccm_c ime r c ime ; 
inC t0,tl,t2,t3; 
cwa__line trap; 

cwa_ calib * calib_i dene icy = NULL.; 
NO_REGISTER(calib_identity) ; 
cu_clear (status, sizeof (cva_status) ) ; 

/♦ CWA_ERR_NO_ I NTERS ECT ION is used =0 signify when we car.-.oc find 

* any edge points for an expected horizontal or vertical line 
*/ 

if Csig = c::_cai=h i0> ; 
goto done ; 

/ * expand the stack to make rccm f-r the 4 : : C icuble arrays w-» 

* allocate 
•/ 

cu_expand_staclc ( 16 ) ; 

if ( 1 result) 

result = kemp ; 

if (limg) 

cct_error ;C3EN_SRR_3ADA?-G ■ ; 

/* If no calibration objec- is passed, make one so -han -here is 

* only one path through the code 
V 

if (: calib) 

calib = calib_identity = cwa_calib_:nake { 0 . , 0 . , 0 - , 1 . . 1 . , 0) ; 
/* 1) use search * / 
ctm_begin ( itime) ; 

ans = gross_locate_crosshair_cenzer_in_image 

(img, (int) (10. * angle* 190 /PI I , igross_pcs_estimate . ; 
if Cans) cct_error (CWA_ERR_NC_INTERSECTJ ,- 

( 

cip_buffer win; 
double x_edge, y_edge ; 
inc edge_val , dir ; 
double angle_of f sec ; 
cct_signal sig; 
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inn i .* 

double left[4]={-2.,-2-,2.,2.}; 
double top(4]«{-2. ,2. .2. . -2. }; 
cip_window ( img, twin, 

groa s_pos_es nirnace . x - 3 5 . gross_pcs_es" zma-e - '/ - 3 5 . TO. ~ 2 ■ .- 

for { i = 0; i < 4 ; i-i-) { 

if (cz_cebt_start_recum_success_val 

(twin. 3, 3, left til . topCiJ , £x_edge, iy_edge. *edge_val. 
idir , &angle_of f set ) ) 
break ; 

status- >concrast_score=cz_map_edge_val_to_tnax_lOOO (edgejvalJ ; 

} 

tO » ctm — read ( &time) ; 

if (cwa_f ind_target_debug_f lag & 0x1* { 
printf ( "gross pas : M ) ; 

cd showas * (ir-c) &gross_pcs_es timate , " cwa_pc:r.: " - . 

} • 

/♦ 2 ) improve the pcs::i=r. ar.c ^r:er.:ac:cr. es:ir,a:a and ccr?.pure 
* the four good rays 



compute_cen-er_posxticn_and_ar.gle_assutr.^r.g_f 2ur_guarar.teed_gcca_r3ys 

(img, calib. &gross _pos_ss:inia:e 1 
transf orni_angle_ac==rding_to_calibrat-ar._map 
( ( (priv_cwa_calib *) calib) - >map. angle) . 
( double ) GUARANTZZD_GCCD_S ZZzL , i improved _pcs_e s c imar e . 
iimproved^hys_orier.tati=n_esti:nate,gccd_rays) ; 



cl = ctm_read ( itime J ; 

if (cwa_f ir.d_target_debug_f lag i 0x1) 
printf ( " improved_pos_es- imate : " ) ; 

cd showas ( (int) &improved_pos_escimace . "cwajjoin^") ; 

} 

/* 3) Enumerate edge pomes along those four good rays and use 
* those lines to compute the final position and orientation estimates 

*/ 

compute_center_posicion_and_angle_along_guaranT:eed_good_rays 

(img, calib, fi t improved__?os_est:imace , gcod_rays , TrGHT_THHESHCLD , 
(cwajjoint *) result , t ( result - >t ) . 
&status->f eature_score) ; 

t2 a ctm_read ( ttime) ; 

cz_transf or^iPoint 

v ( (priv_cwa_calib * J calib j - >map, 
result- >x, result- >y , 
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&result->x. iresult- >y) ; 

if (cwa_f ind_target_debug_f lag & 0x1) { 
printf ("result: ■) ; 

cd_3howas ( ( int ) resul t , ** cwa^ point M ) ; 

} 

if (cwa_f ind_target_debug_f lag & 0x4 ) { 
printf ( "Times : " ) ; 

printf (". gross locate: %d\n" . tO) ; 
printf (" fine locate: Vd\n" . tl-tO) ; 
printf (" finer locate: Vd\n" , z2 - tl) ,- 

} 

done : 

cu_expand_stack (-16) ; 

if (calib_identity) cwa_calib_delete (caiib_identity; ; 

/* Make returned angle agree wi:h given angle argument v 

if (sig) cct_throw(sig) ; 

for (i =» -8; i <= 3; i*-) 
if { (i != 0) && 

(fabs (result- >t - i*?Z.'2 - angle ) < fabs .result - >z - angle 
result. - >c •»-= i*PI/2 , i - - ; 

return 1; 

} 

int cwa_f ind_target 

(cip_buffer »img, cva_calib *cal-b, double ancle, cwa_lir.e -icSul:, 
cwa_ status * status ) 

( 

cwa_status tmp_status; 
cct_signal sig; 

if (1 status) status - &cmp_scacus; 

cc t_ca t ch ( CGEN_ERR_BADARG ) ; 
cct~catch(CWA_ERR_BACKLIT) ; 
cct_catch(CWA_ERR_TRACKER) ; 

if (sig=cct_cat:ch(0) ) { 

if (sig == CGEN_ERR_BADARG) { 

status- >error_f lags i = CWA_ERR_3ADARG ; 
cct_end(sig) ; 
cct_error (sig) ; 

} 

status- >error_f lags != CMA_ERR_FAILED_TC_FIND_TARGET; 
return status- > found = 0; 

} 

cwa f ind_target_innernal (img, calib, angle, result . status) ; 
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return status - > found =* Ij 
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Described herein are calibration targets and methods for detei*mining the location 
and orientation thereof meeting the objects set forth above. It will be appreciated that the 
embodiments shown in the drawings and discussed above are merely illustrative of the 
invention, and that other embodiments incorporating changes therein fall within the scope 
of the invention, of which we ciaim: 
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1 . A machine vision method of determining a location of a reference point in an 
image of a calibration target, the method comprising the steps of 

A. generating an image of a target that includes 

5 

i. two or more regions, each region being defined by at least two linear 
edges that are directed toward a reference point; 

ii. . at least one of the regions having a different imageable characteristic 
10 from an adjacent region; 

B. identifying edges in the image corresponding to lines in the calibration target; 

C. determining a location where lines fitted to those edges intersect. 

15 

2. A method according to step 1, wherein step (B) comprises the step of identifying 
edges in the image by applying an edge detection vision tool to the image. 

3. A method according to step 1, wherein step (B) comprises identifying edges in the 
20 image by applying a caliper vision tool to the image. 

4. A method according to step 1, wherein step (B) comprises identifying edges in the 
image by applying a first difference operator vision tool to the image, and by 
applying a peak detector vision tool the an output of the first difference operator 

25 vision tool 

5. A method according to step 1, wherein step (B) comprises identifying edges in the 
image by applying a caliper tool to the image beginning at an approximate location 
of the reference point. 

30 
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6. A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a Hough line vision tool to the image 
and finding an intersection of lines identified thereby. 

5 7. A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a correlation vision tool to the image 
using a template substantially approximating an expected pattern of edges in the 
image. 

10 8. A method according to claim 5, comprising the step of determining an approximate 
location of the reference point by applying a projection vision tool to the image 
along one or more axes with which the edges are expected to be aligned. 

9. A method according to claim 1, comprising the steps of 

15 

identifying edges in the image corresponding to lines in the calibration target; and 

determining an orientation of the target based on the angle of those identified 
edges. 



20 



25 



30 



10. A machine vision according to claim 1, wherein 

step (B) includes the step of applying a Hough line vision tool to identify edges in 
the image corresponding to lines in the calibration target; and 

the method further including the step of determining an orientation of the target 
based on the angle of those identified edges. 

11. A machine vision according to claim 1, comprising 

A. applying a Sobel edge tool to the image to generate at least a Sobel angle image; 
and 
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B. taking an angle histogram of the 'Sobel angle image to determine an orientation of 
the target. 



12. A machine vision method of an orientation of a calibration target in an image, the 
method comprising the steps of 

A. generating an image of a target that includes 

L two or more regions, each region being defined by at least two linear 

edges that are directed toward a position; 

ii.. at least one of the regions having a different imageable characteristic 
from an adjacent region; 

B. identifying edges in the image corresponding to lines in the calibration target; 

C. determining an orientation of the target based on the angle of those identified 
edges. 

13. A method according to claim 12, wherein 

step (B) includes applying a Hough line vision tool to the image to identify therein 
edges in the image corresponding to lines in the calibration target; and 

step (C) includes determining an orientation of the target based on the angle of 
those identified edges. 

14. A method according to claim 12, wherein 

step (B) includes applying a Sobel edge tool to generate at least a Sobel angle 
image; and 
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step (C) includes taking an angle histogram of the Sobel angle image to determine 
an orientation of the target. 

15. A method according to any of claims 1 and 12, wherein step (A) includes the step 
5 of generating an image of a target in which each region has any of a different 

color, contrast, brightness and stippling from regions adjacent thereto. 

16. A method according to claim 1 5, for use with a machine vision system having 
image capture means for capturing an image of the target, wherein each region has 

10 a different characteristic in such an image than the regions adjacent thereto. 

17. A method according to any of claims 1 and 12, wherein step (A) includes the step 
of generating an image of a target for which the reference point is at a center 
thereof. 

15 

18. A method according to claim 17, wherein step (A) includes the step of generating 
an image of a target for which the linear edges substantially meet at the reference 
point. 

20 19. A method according to any of claim 1 and 12, wherein step (A) includes the step of 
generating in image of a target having four imageable regions. 

20. A method according to claim 19, wherein step (A) includes the step of generating 
an image of a four-region target in which the at least two linear edges of each of 

25 the regions are perpendicular to one another. 

21 . A machine vision method of determining a location and orientation of a calibration 
target in an image thereof, the method comprising the steps of 

30 A. generating an image of a target that includes 
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i. two or more regions, each region being defined by at least two linear 
edges that are directed toward a reference point; 

ii. . at least one of the regions having a different imageable characteristic 

from an adjacent region; 

B. generating, based on analysis of the image, an estimate of an orientation of the 
target therein; 

C. generating, based on analysis of the image, an estimate of a location of the 
reference point therein; 

D. refining, based on analysis of the image, estimates of at least one of the location of 
the reference point in the image and the orientation of the target in the image. 

22. A method according to claim 21, wherein step (B) includes one or more of the 
following steps: 

i. applying a Hough line vision tool to the image to find an angle of edges 
in the image; 

ii. applying a Sobel edge tool to the image to generate Sobel angle image, 
generating a histogram of the angles represented in that Sobel angle 
image, and determining a one dimensional correlation thereof; and 

iiL inputting an orientation from the user 

23. A method according to claim 22, wherein step (C) includes one or more of the 
following steps: 



124 



SUBSTITUTE SHEET (RULE 26) 



WO 98/18117 



PCT/US97/18268 



i. applying a Hough vision tool to the image to find an approximate 

location of edges therein, and finding an intersection of lines defined by 
those edges; 

5 ii. applying a projection tool vision tool to the image to find an approximate 

location of edges therein, and finding an intersection of lines defined by 
those edges; 

iil applying a correlation vision tool to the image to find approximate 
10 coordinates of the reference point; and 

iv. determining a sum of absolute differences to find approximate 
coordinates of the reference point. 

15 24. A method according to claim 23, wherein step (D) includes the step of invoking 
the steps in sequence, at least one time: 

i. applying a caliper vision tool along each of the expected edges in the 
image, where those edges are defined by a current estimate of the 

20 reference point location and a current estimate of the target orientation; 

and 

ii. fitting lines to edge points determined by the caliper vision tool. 

25 25. A method according to claim 24, wherein step (D)(i) includes the step of applying 
the caliper vision tool along each of the expected edges until any of the following 
conditions is met: no edge is found by the caliper vision tool at a sample point; 
more than one edge is found by the caliper vision tool at a sample point; a distance 
between a sample edge point and a nominal line is larger than a threshold; or, the 

30 caliper vision tool window for the sample edge point extends outside of the image. 
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